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ABSTRACT 


Progress  on  real-time  1-D  and  2-D  spatial  light  modulators,  optical  systolic 
array  processors,  optical  image  processing,  and  distortion-invariant  pattern 
recognition  are  reported.  Our  real-time  spatial  light  modulator  research  con¬ 
cerns  the  Soviet  PRIZ  device  and  bulk  acousto-optic  transducers.  Novel  features 
detailed  for  the  Soviet  device  include:  uniform  and  directional  spatial  filter¬ 
ing  and  static  image  suppression  or  change  detection.  Systolic  array  processor 
research  includes  a  new  frequency-multiplexed  architecture,  realization  of 
singular  value  decomposition  and  matrix  decomposition  algorithms.  Optical  image 
processing  and  pattern  recognition  research  received  major  attention.  A  new 
class  of  nonlinear  local  operators  including  the  Sobel  operator  were  described 
and  demonstrated.  New  optical  feature  generation  techniques  for  distortion- 
invariant  pattern  recognition  were  developed.  These  included  the  generalized 
optical  chord  transformation.  Our  primary  distortion-invariant  pattern  recog¬ 
nition  research  addressed  synthetic  discriminant  functions  with  attention  to 
two  new  efficient  calculation  techniques.  The  performance  of  synthetic  dis¬ 
criminant  functions  for  multi-class  distortion-invariant  pattern  recognition, 
and  initial  noise  performance  of  this  optical  pattern  recognition  algorithm. 
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1.  INTRODUCTION 

During  the  past  year  (September  1982  -  September  1983) ,  our  research  in 
optical  data  processing  for  missile  guidance  has  addressed  many  of  the  key 
issues  and  aspects  of  this  technology.  This  research  includes:  various  new 
devices  and  components,  new  system  architectures,  new  high-speed  general  pur¬ 
pose  optical  data  processing  techniques  and  systems,  tests  on  new  image  data¬ 
bases,  basic  studies  of  existing  pattern  recognition  architectures,  and  new 
pattern  recognition  techniques,  algorithms,  and  concepts.  As  in  past  years, 
we  have  been  quite  faithful  in  reporting  our  AFOSR  sponsored  research  in  vari¬ 
ous  journals  and  conference  publications.  Copies  of  the  more  relevant  papers 
we  have  published  over  the  past  year  are  included  as  the  chapters  of  this 
report  to  provide  complete  documentation  of  each  aspect  of  our  work. 

In  Section  2,  we  provide  a  summary  and  overview  of  our  research  progress 
achieved  during  the  past  year.  This  work  addresses  six  vital  areas  of  optical 
data  processing  research:  (1)  real-time  spatial  light  modulators  (Sections  3 
and  4) ;  (2)  systolic  array  processors  (Sections  5-8) ;  (3)  image  preprocessing 
(Section  9);  (4)  optical  correlator  analysis  (Sections  10-11);  (5)  optical 
feature  extraction  (Section  12) ;  and  (6)  synthetic  discriminant  functions  (Sec¬ 
tions  13-15).  Topic  (1)  concerns  the  vital  issue  of  real-time  devices;  topic 
(2)  concerns  the  hottest  topic  in  optical  processing  at  present  and  a  potentially 
quite  general-purpose  optical  processor;  topic  (3)  applies  parallel  optical  tech¬ 
niques  to  achieve  operations  normally  performed  by  digital  image  preprocessors; 
topic  (4)  provides  a  better  understanding  of  conventional  optical  pattern  rec¬ 
ognition  architectures  with  attention  to  the  performance  obtained  from  these 
systems;  topics  (5)  and  (6)  address  distortion-invariant  optical  pattern  recog¬ 
nition  techniques  (feature  extraction  and  correlation). 


Details  on  the  more  salient  results  of  our  research  are  provided  in  Sec¬ 
tions  3-15.  In  Section  16,  we  enumerate  our  AFOSR  sponsored  publications,  the 
presentations  given  on  this  research  at  conferences  and  seminars  during  the 
past  year,  and  the  Master's  and  PhD  students  that  this  grant  has  supported. 

During  the  past  year,  the  principal  investigator  (PI)  presented  invited 
talks  on  our  AFOSR  sponsored  research  at  the  following  conferences:  SPIE  Inter¬ 
national  Conference  (San  Diego,  August  1983),  IOCC  Conference  (Boston,  April 
1983),  and  CLEO'83  Conference  (Baltimore,  May  1983).  The  PI  also  chaired  con¬ 
ference  sessions  and  seminars  and  served  on  the  organizing  committees  for  the 
following  conferences  and  topics:  SPIE  (Robotics),  CLEO  (Lasers  and  Light 
Modulators),  and  IOCC  (Optical  Computing). 


-3- 


2.  SUMMARY  AND  OVERVIEW 

Sections  3-15  detail  our  recent  research  on  seven  different  aspects  of 
optical  data  processing.  Brief  highlights  of  each  of  these  sections  and  topics 
follow  below. 

2.1  REAL-TIME  SPATIAL  LIGHT  MODULATORS 

Real-time  spatial  light  modulators  are  the  primary  elements  necessary  for 
the  realization  of  optical  pattern  recognition  systems.  Our  research  during 
the  past  year  on  two  different  types  of  these  devices  follow.  The  details  of 
this  work  are  provided  in  Sections  3  and  4  of  this  report. 

2.1.1  Soviet  PRIZ  Light  Modulator  (Section  3):  A  unique  opportunity  arose  for 
us  to  conduct  a  first-hand  four  month  test  and  evaluation  of  a  Soviet  spatial 
light  modulator  (the  PRIZ).  Section  3  is  the  last  paper  on  this  topic.  It  sum¬ 
marizes  our  research,  the  performance  of  these  devices,  their  applications,  and 
their  unique  features  (these  include  automatic  edge  enhancement,  directional 
edge  enhancement,  and  dynamic  image  subtraction).  Further  U.S.  research  is 
necessary  on  these  devices  to  facilitate  fabrication  of  equivalent  U.S.  com¬ 
ponents  and  to  allow  full  understanding  of  the  operation  of  these  new  devices. 

2.1.2  Acousto-Optic  Light  Modulators  (Section  4):  The  most  commercially  avail¬ 
able,  proven  and  reliable  spatial  light  modulators  are  acousto-optic  cpIIs.  A 
summary  of  these  elements,  the  basic  architectures  employing  them,  and  several 
new  architectures  and  applications  employing  these  components  are  reviewed  in 
Section  4.  These  include  systems  to  process  long  codes  and  systems  less  suscep¬ 
tible  to  various  acousto-optic  device  shortcomings.  These  elements  are,  in 
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general,  most  appropriate  for  signal  rather  than  image  processing  and  are  quite 
attractive  for  optical  systolic  processor  fabrication. 

2.2  OPTICAL  SYSTOLIC  ARRAY  PROCESSORS  (SECTIONS  5-8) 

One  of  the  most  general-purpose  and  flexible  optical  data  processing  tech¬ 
niques  is  the  optical  systolic  array  processor.  These  optical  architectures 
perform  matrix-vector  and  various  linear  algebraic  operations  optically  with 
high-speed,  parallel  processing,  and  very  high  computational  rates  possible. 
Sections  5-8  detail  our  recent  research  in  this  area.  This  work  is  intended  to 
develop  an  optical  systolic  processor  for  air-to-air  missile  guidance  and  control. 

2.2.1  Frequency-Multiplexed  Optical  Systolic  Processor  (Section  5):  A  new 
optical  systolic  array  architecture  using  acousto-optic  cells  was  devised  and 
described.  This  system  employs  frequency-multiplexing  to  represent  2-D  data. 

We  refer  to  it  as  a  frequency-multiplexed  acousto-optic  systolic  array  proces¬ 
sor.  Specific  attention  was  given  to:  a  new  matrix  inversion  and  linear  alge¬ 
braic  equation  solution  algorithm,  a  new  technique  for  handling  bipolar  data  and 
new  matrix-matrix  and  matrix-matrix-matrix  multiplication  algorithms.  Primary 
attention  was  also  given  to  the  efficient  flow  of  data  and  operations  on  such 

a  processor  and  to  the  demonstration  that  this  one  system  could  achieve  all  of 
the  basic  operations  required  in  Kalman  filtering.  The  method  that  we  are  pur¬ 
suing  for  our  air-to-air  missile  guidance  problem  employs  a  Kalman  filter.  We 
are  working  in  conjunction  with  AFIT  researchers  in  this  area. 

2.2.2  Direct  Matrix  Solutions  (Section  6):  Optical  systolic  array  processors 
allow  direct  matrix  decomposition  to  be  efficiently  performed.  Optical  matrix- 
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vector  processors  cannot  efficiently  perform  this  operation.  We  were  the  first  i 

to  address  this  topic  and  to  note  that  the  major  computational  load  in  a  direct 

solution  was  the  matrix  decomposition  itself.  We  thus  considered  techniques  to 

achieve  matrix  decomposition  on  our  frequency-multiplexed  optical  systolic  array 

processor.  We  detailed  how  to  realize  the  LU  and  Cholesky  decomposition  on  our 

optical  processor.  This  included  a  new  parallel  LU  algorithm  and  full  detailing 

of  the  flow  of  data  and  operations  in  the  optical  version  of  this  algorithm. 

New  attention  to  a  parallel  algorithm  was  required  for  efficient  realization  of 
this  technique  on  an  optical  processor. 

2.2.3  Optical  Singular  Value  Decomposition  (Section  7):  Singular  value  decom¬ 
position  is  an  attractive  (and  in  many  cases  an  essential)  technique  for  solving 
matrix  equations,  inverting  matrices  and  for  singular  matrix  processing.  We 
detailed  one  technique  to  achieve  this  operation  on  a  general  matrix-vector  pro¬ 
cessor. 

2.2.4  Guidelines  (Section  8):  Considerable  interest  and  many  publications 

exist  on  optical  systolic  processors.  However,  much  of  the  published  work  is 

not  properly  directed.  In  this  section,  we  note  that  attention  should  be  given: 

3 

to  N  problems,  to  architectures  that  achieve  flexible  performance,  to  architec¬ 
tures  that  do  not  require  parallel  output  2-D  detector  arrays,  to  architectures 
that  avoid  optical  system  sources  and  to  systems  that  can  be  easily  fabricated 
and  realized  today. 

2.3  OPTICAL  IMAGE  PROCESSING  (SECTION  9) 

We  distinguish  between  optical  image  processing  and  optical  pattern  rec¬ 
ognition.  In  image  processing,  the  input  to  the  system  is  an  image  and  so  is 


the  output  (usually  an  enhanced  image).  Our  new  research  includes  the  first 
demonstration  of  an  optical  nonlinear  local  operator.  Such  operations  are 
normally  performed  digitally  and  used  for  image  preprocessing  or  image  proces¬ 
sing.  We  describe  two  optical  architectures  to  achieve  arbitrary  nonlinear 
local  operators  of  large  size  with  the  parallelism  and  speed  of  optical  systems 
fully  utilized.  These  architectures  include  correlators  using  multiple  matched 
spatial  filters  and  computer  generated  holograms.  The  nonlinear  local  operator 
we  detailed  and  demonstrated  was  the  Sobel  operator.  This  edge-enhancement 
operator  is  most  attractive  and  necessary  in  infrared  and  multisensor  image 
preprocessing.  Extensions  of  this  new  basic  technique  are  possible  and  should 
be  pursued. 

2.4  OPTICAL  CORRELATOR  ANALYSIS  (SECTION  10) 

Optical  correlators  are  well-known  and  often  used.  In  our  work,  we  addressed 
two  potential  error  sources  in  optical  correlators  and  especially  in  optical 
correlators  in  which  coefficient-estimation  is  the  purpose  of  the  optical  ar¬ 
chitecture.  Those  error  sources  considered  include:  finite  space  bandwidth 
product  and  global  rather  than  zero-mean  data  correlation.  The  first  issue  is 
important  when  the  statistical  correlation  features  are  optically  estimated. 

We  show  from  the  standard  deviation  of  the  crosscorrelation  coefficient  that 
the  error  in  the  correlation  estimate  can  be  appreciable  if  the  space  bandwidth 
product  of  the  image  data  is  small.  Major  attention  is  given  to  the  second 
possible  error  source,  since  it  is  not  apparently  well-known  within  the  optics 
community.  Conventional  optical  correlators  automatically  suppress  dc  spatial 
frequency  data  and  thus  correlate  zero-mean  data.  This  represents  no  problem  if 
the  location  of  the  correlation  peak  rather  than  the  correlation  peak  value  is 


of  major  concern.  When  the  correlation  peak  value  itself  is  the  parameter  of 
concern,  then  one  must  carefully  distinguish  between  local  and  global  zero-mean 
data  being  used,  if  one  is  to  achieve  an  accurate  correlation  peak  estimate. 
Quantitative  data  on  the  magnitude  of  each  of  these  effects  is  included. 

2. 5  BINARY  IMAGE  CORRELATIONS  (SECTION  11) 

As  an  extension  of  earlier  AFOSR  research,  we  experimentally  considered  the 
performance  of  correlators  operating  on  binary  images.  Such  architectures  are 
attractive  for  digital  and  optical  implementation,  because  of  the  reduced  dy¬ 
namic  range  that  is  needed.  Using  the  peak-to-sidelobe  ratio  of  the  correlation 
output,  we  show  in  theory  and  confirm  by  experiments  that  a  binary  correlator 
(i.e.  a  correlator  operating  on  binary  or  two-level  image  data)  can  provide 
better  performance  than  a  correlator  operating  on  gray-scale  imagery. 

2.6  OPTICAL  FEATURE  EXTRACTION  (SECTION  12) 

In  this  year,  we  introduced  a  major  new  approach  to  distortion-invariant 
pattern  recognition:  the  hybrid  combination  of  optical  feature  generation, 
digital  feature  extraction  and  classification.  In  Section  12,  we  advance  a  new 
feature  set  (the  generalized  chord  distribution),  a  new  optical  technique  to 
produce  these  object  features  and  the  use  of  a  digital  classifier  that  maximizes 
the  Fisher  ratio.  This  digital  classifier  is  used  for  feature  extraction  and 
for  analysis  of  the  optically  generated  feature  outputs.  We  demonstrated  this 
technique  for  distortion-invariant  multi-class  pattern  recognition  and  obtained 
very  promising  results.  We  plan  further  work  on  this  technique  and  other  opti- 
ally-generated  features. 
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2.7  SHIFT-INVARIANT  DISTORTION-INVARIANT  PATTERN  RECOGNITION  (SECTIONS  13-15) 

All  feature  extractors  (optical  or  digital)  cannot  accommodate  multiple 
objects  or  significant  noise.  Only  correlators  can  achieve  such  multi-object 
or  shift-invariant  performance  with  high  noise  or  clutter  present.  The  major 
approach  we  have  pursued  for  this  purpose  is  the  synthetic  discriminant  function 
concept.  In  this  case,  a  synthetic  discriminant  function  (capable  of  distortion- 
invariant  pattern  recognition)  is  synthesized  off-line  and  then  an  optical 
matched  spatial  filter  of  it  is  formed  and  used  in  an  optical  or  digital  cor¬ 
relator  . 

2.7.1  Efficient  Matrix  Inversion  (Section  13):  Our  off-line  synthetic  dis¬ 
criminant  function  synthesis  techniques  require  processing  and  inversion  of 
large  matrices.  In  Section  13,  we  describe  a  new  and  most  efficient  technique 
we  devised  to  achieve  this  with  significantly  reduced  storage  and  computations. 
This  is  quite  essential  and  necessary  as  the  size  of  the  synthetic  discriminant 
function  training  set  increases.  The  algorithm  we  developed  is  essentially  an 
efficient  matrix  inversion  technique  with  minimal  data  storage  requirements. 
Experimental  comparison  on  tank  images  was  used  to  verify  the  superiority  of 
this  new  off-line  filter  synthesis  technique. 

2.7.2  Synthetic  Discriminant  Function  Performance  (Section  14):  We  have  ob¬ 
tained  a  large  four-class  database  of  ship  images  with  36  different  distorted 
versions  of  each  ship  class  available  (taken  at  10°  intervals  around  the  object 
from  a  zero-degree  attack  angle).  In  Section  14,  we  provide  quantitative  data 
on  the  first  full  test  of  our  four  different  types  of  synthetic  discriminant 
functions  on  this  144  image  data  base.  The  performance  obtained  is  quite 


excellent  with  over  90%  correct  classification  achieved  for  multi-class  dis¬ 


tortion-invariant  pattern  recognition. 

2.7.3  Initial  Noise  Performance  (Section  15):  In  Section  15,  we  review  our 
synthetic  discriminant  function  synthesis  techniques  and  advance  the  first  noise 
performance  of  these  algorithms.  The  performance  obtained  in  the  presence  of 
noise  was  excellent.  These  are  perhaps  the  most  impressive  and  comprehensive 
pattern  recognition  results  (optical  or  digital)  obtained  for  distortion- 
invariant  multi-class  pattern  recognition. 


3.  APPLICATIONS  OF  THE  PRIZ  LIGHT  MODULATOR 
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Applications  of  the  Priz  light  modulator 


David  Casasent,  Frank  Caimi,  M.  P.  Petrov,  and  A.  V.  Khomenko 


The  Priz  light  modulator  suppresses  input  data  at  zero  spatial  frequency,  can  provide  directional  spatial  fil¬ 
tering,  and  can  perform  dynamic  image  selection  or  change  detection.  In  this  paper,  we  summarize  the 
Friz's  performance  and  provide  experimental  confirmation  of  the  above  three  image  processing  applications 
of  this  device. 


I.  Introduction 

Optical  signal  processing  and  image  pattern  recog¬ 
nition  applications  require  real-time  and  reusable  de¬ 
vices  on  which  the  input  data  to  be  processed  can  be 
recorded  for  subsequent  optical  processing.  These  vital 
components  in  an  optical  processor  are  known  as  spatial 
light  modulators  (SLM).  Although  many  candidate 
SLM  devices  exist,1  we  will  restrict  our  present  atten¬ 
tion  to  only  one  such  device,  the  Priz  light  modulator.-  K 
Priz  is  a  Soviet  acronym  that  translates  as  image 
transformer.  This  modulator  was  proposed  by  a  group 
of  researchers  of  A.F.  Ioffe  Physico-Technical  Institute 
(FTI)  of  the  Academy  of  Sciences  of  the  U.S.S.R.  It 
employs  the  same  active  element,  i.e„  the  bismuth  sil¬ 
icon  oxide  (BSO)  type  crystal,  as  the  well-known  Prom 
modulator.9 

At  the  Electrical  Engineering  Department  of  Car- 
negie-Mellon  University  (CMU),  five  Prom  and  Priz 
units  fabricated  at  the  FTI  laboratory  headed  by  Petrov 
were  tested  and  evaluated.  One  worker  of  FTI  partic¬ 
ipated  in  the  research  program  (Khomenko). 

In  Sec.  II,  we  review  the  structure  of  the  Priz  light 
modulator  and  the  motivation  for  its  fabrication  and 
highlight  the  spatial  frequency  response  data  obtained 
on  the  devices  we  evaluated  at  CMU.  We  then  include 
(Sec.  Ill)  a  summary  of  the  dynamic  and  optical  per¬ 
formance  of  the  Priz.  These  data  were  obtained  from 
experiments  performed  in  both  the  Soviet  Union  and 
at  CMU.  We  include  several  image  and  signal  pattern 
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recognition  correlation  examples  of  the  use  of  the  device 
in  Sec.  IV.  Because  of  the  transverse  linear  electrooptic 
effect  used  in  the  Priz  to  modulate  read  light,  it  exhibits 
three  unique  features  that  are  of  use  in  various  image 
processing  and  pattern  recognition  applications.  These 
include;  suppression  of  dc  and  low  spatial  frequency- 
data  as  well  as  directional  filtering  of  input  spatial 
frequencies  oriented  in  selected  angular  directions  (Sec, 
V)  and  a  quite  unique  feature  referred  to  as  dynamic 
image  selection  in  which  the  device  responds  only  to 
changes  in  the  input  image  data  (Sec.  IV).  Our  sum¬ 
mary  and  concluding  remarks  follow  in  Sec.  VII. 

11.  Operation  of  the  Priz 

The  Prom  light  modulator9  consists  of  a  BSO  crystal 
~2()  X  20  X  0.4  mm  with  Parylene  insulating  layers  and 
transparent  electrodes  on  the  large  faces.  In  operation 
the  spatially  modulated  data  to  be  processed  are  imaged 
or  scanned  onto  the  device  in  Aw  write  light  (350-450 
nm).  Photocarriers  are  generated  in  the  photocon- 
ductive  BSO,  and  a  spatially  modulated  charge  layer  is 
produced  within  the  BSO.  When  the  device  is  illumi¬ 
nated  with  a  uniform  read  light  beam  at  \n  (usually  633 
nm),  the  A/<  light  emerging  from  the  device  is  polariza¬ 
tion  modulated  spatially  with  an  amplitude  of  modu¬ 
lation  that  varies  spatially  in  accordance  with  the 
original  Aiv  input  light  or  data  pattern.  This  A/,>  mod¬ 
ulation  occurs  by  the  linear-longitudinal  electrooptic 
or  Pockels  effect.  The  polarization  modulation  can  be 
converted  to  amplitude  modulation  when  a  crossed 
analyzer  is  placed  behind  the  modulator. 

In  the  Priz  light  modulator,-  H  a  1 1 10]  or  1 1 1 1  ]  cut 
BSO  crystal  is  used  rather  than  the  [100]  cut  crystal 
used  in  the  Prom.  Other  proprielary  fabrication 
techniques  are  employed,  but  the  issue  of  major  im¬ 
portance  is  that  with  these  different  crystal  cuts,  the 
device  now  modulates  A«  light  by  the  transverse  rather 
than  the  longitudinal  electrooptic  effect.  The  spatially 
varying  Aw  light  distribution  is  still  incident  on  the 
crystal’s  large  faces  collinear  with  the  applied  electric 
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field  direction,  and  the  spatially  varying  charge  layer 
parallel  to  the  crystal’s  large  faces  is  still  induced. 
However,  the  transverse  component  of  this  field  is  what 
is  used  to  provide  the  spatial  modulation  of  the  Xu 
light. 

The  Prom  exhibits  low  diffraction  efficiency  ij  and  a 
sharp  t)  ~  1//'  decrease  in  usable  output  light  intensity 
at  high  spatial  frequencies/.1"  If  we  assume  that  the 
sine  wave  electric  charge  grating  with  amplitude  <j„  in¬ 
duced  during  image  writing  in  the  Prom  is  infinitely  thin 
and  that  it  exists  at  the  crystal  dielectric  interface,  the 
phase  modulation  vs  spatial  frequence  is  described 
by" 

So  =  -7 — : - : - : —  •  ill 

/  v  J\tlt  rolh-jr/i/,/  +  <.  mth 2ir/t/.  ) 

where  (tt  and  <,  are  the  relative  dielectric  constants  of 
the  dielectric  layer  and  the  BSO  crystal,  respectively, 
dti  and  d,  are  the  thicknesses  of  these  layers,  and  l  \  L. 
is  the  halfwave  voltage  of  the  crystal.  Kquation  (1) 
indicates  that  when  /  increases  at  /  >  'hd,i,  coth2 irfd.i 

1  and  coth2,T/<7,  1,  and  thus  Ai/>  decreases  )//. 

Since  1)  *  (A</>)J,  Kq.  (1)  predicts  i]  a  1  If-.  However,  in 
experiments  a  sharper  (r\  1  //')  dependence  was  ob¬ 
served.  This  was  attributed  to  the  fact  that,  in  the 
process  of  image  writing,  a  volume  electric  charge  dis¬ 
tribution  is  formed  within  the  crystal  volume  rather 
than  an  infinitesimally  thin  one.1"  I  ’sing  this  new 
model,  it  has  been  shown  that  in  the  case  when  the 
charge  is  distributed  throughout  a  layer  of  thickness  d„ 
near  the  crystal  dielectric  interface 

iT|ijci)sh’j7r/'</,  -  t-osh'.’ir/W, 

jk«/>  = - 77 - •  (2) 

(  \  cuth'-’ir/d,/  +  <,  cothgir/i/,  1  •  siiih'Jir/f/. 

Equation  (2)  predicts  an  rj  *  l/f  '  dependence  at  high  /, 
which  agrees  with  experimental  results.  Thus  the  sharp 
dependence  of  17  on  /  for  the  Prom  is  attributable  to  the 
volume  character  of  charge  distribution.  The  most 
complete  and  detailed  theoretical  description  of  the 
Prom  device  can  be  found  in  Refs.  12  and  12.  These 
papers  discuss  a  model  that  includes  the  location  of  the 
charge  layer  within  the  BSO,  the  thickness  of  the  dif¬ 
ferent  device  layers,  the  wavelength  of  the  light  used. 
To  overcome  the  disadvantages  of  the  volume  charge 
predicted  by  Eq.  (2),  it  was  suggested  in  Refs.  14  and  lf> 
to  use  the  transverse  electrooptic  effect  for  read  light 
modulation  rather  than  the  longitudinal  effect  used  in 
the  Prom.  The  modulator  that  uses  the  transverse 
electrix>ptic  effect  is  called  the  Priz.  The  corresponding 
calculation  of  the  phase  modulation  for  the  case  of  the 
transverse  effect  shows  that" 

^  ” "n  1 1 _ <rf|cosh2ff/d|  +  i'osh'2jr/(rf,.  -  d„)  —  1 1  +  ,  j 

t  v.’O  t  1  2  7r  j  'da  I  t ,t  t anh2 n  jii,  I  taiih'J )  i-osh2 irfd,  j 

where  k  \  =  r,  tanh2?r /drf  sinh2jr/d„. 

From  this  formula,  two  characteristic  features  of  the 
Priz  device  can  be  seen.  First,  A</>(0)  =  0;  i.e.,  the 
modulator  suppresses  the  dc  component.  Second,  at 
high  /,  A  ip  a  1//  and  rj  a  1  i.e.,  the  Priz  tj  vs  /  char¬ 

acteristic  is  superior  to  that  of  the  Prom. 


III.  Performance  of  the  Priz 

In  the  MTF  tests  performed  on  the  Priz  at  ('Ml  ',  we 
operated  the  device  at  2  frames/sec.  In  other  experi¬ 
ments,  we  operated  the  device  at  20  frames/sec.  A 
faster  frame  rate  should  be  possible,  but  no  effort  has 
vet  been  made  to  determine  the  device's  maximum  cycle 
time.  However,  results  of  several  experiments  are 
useful  in  providing  some  indication  of  the  final  device 
performance  possible  on  the  Priz. 

In  the  experiments  performed  in  FT1,  the  device  was 
operated  with  write  times  as  short  at  7  nsec  using  a 
pulsed  laser  source  with  an  intracavity  electrooptic 
modulator.  In  this  case,  photocarriers  were  generated 
in  a  negligible  time,  but  the  output  light  pattern  was  not 
visible  until  1  n sec  later,  and  it  peaked  after  10  /jsec. 
Carrier  mobility  and  transit  times  thus  appeared  to 
limit  the  minimum  write  read  cycle  times  of  the  Priz 
to  10/zsec.  In  the  more  conventional  operating  mode. 
1  -msec  exposure  times  are  used,  and  the  output  pattern 
is  then  immediately  visible. 

Erase  time  is  a  second  limitation  on  the  device's 
speed.  In  all  tests  performed  at  CMC,  a  fixed  1  -msec 
erase  flash  (from  the  standard  erase  unit  provided  with 
the  C.S.A.  Prom)  was  used.  However,  neither  the  Prom 
nor  the  Priz  can  be  recycled  immediately  after  erasure, 
and  a  delay  time  is  necessary  to  allow  redistribution  and 
relaxation  of  excited  carriers  within  the  crystal.  One 
millisecond  of  relaxation  time  and  hence  a  total  1-msec 
erase  time  appear  adequate.  High-energy  erase  pulses 
cannot  decrease  this  time,  since  they  generate  and  dis¬ 
lodge  other  carriers  within  the  bulk  of  the  device,  and 
longer  relaxation  times  between  the  end  of  erasure  and 
the  start  of  a  new  write  cycle  then  become  necessary. 
Thus  operation  of  the  Priz  at  a  l():i-frame/sec  (write 
read  -erase  cycle)  rate  appears  possible,  but  additional 
theory  and  experiments  are  necessary  to  confirm  this. 
Moreover,  an  application  for  which  the  entire  spatial 
input  data  changes  every  millisecond  is  necessary  to 
merit  such  an  effort  together  with  attention  to  how  one 
can  introduce  such  a  new  2-1)  \n  spatial  distribution 
to  the  device  at  these  10:,-frame/see  rates. 

The  storage  times  for  the  Priz.  are  adequate  for  most 
applications  |l-min  storage  in  the  dark  and  10  20-sec 
storage  under  a  high  Xu  =  (52.1-nm  read  light  intensity 
(I u  =  -  mW/cm)|.  These  can  be  somewhat  controlled 
by  varying  the  thickness  of  the  insulating  layers.  The 
lifetime  of  the  Priz,  like  that  of  the  Prom,  appears  to  be 
excellent.  Selected  Priz  units  have  been  operated  for 
over  one  million  cycles  at  a  20-frame/sec  rate  with  no 
noticeable  change  in  performance.  The  only  concern 
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with  the  device’s  lifetime  appears  to  he  its  operation 
with  high  In  light  levels.  In  this  case,  a  device  with  high 
transmittance  and  electrodes  with  good  conductivity 
is  necessary.  For  such  cases,  lnO_>  electrodes  are  used. 
The  performance  of  such  Friz  units  appears  to  he  good 
(as  several  of  them  have  operated  successfully  for  sev¬ 
eral  years). 

The  resolution  of  the  Friz  is  mostly  understood  and 
has  been  experimentally  verified  by  several  techniques. 
Its  diffraction  efficiency  ij ,>  at  low  spatial  frequencies 
is  MCc,  its  10-dB  spatial  frequency  response  /(U  is  ~20 
cvcles/mm,  and  the  spatial  frequency  fiun  at  which  i)  = 
0.01  is  100  cvcles/mm.  At  CMU,  we  operated  the 
device  with  input  data  having  spatial  frequencies  as  high 
as  80  cycles/mm.  With  higher  quality  optical  systems 
and  a  different  area  detection  technique,  FTI  re¬ 
searchers  have  measured  diffraction  efficiency  on  the 
device  beyond  500  cycles/mm.  Since  the  usable  output 
light  intensity  at  these  high  spatial  frequencies  is  quite 
low,  only  in  selected  applications  can  such  resolution 
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Fig.  I.  friz  spatial  light  modulator. 


actually  be  used.  The  Friz  performs  well  with  input 
write  light  exposures  f„V  of  50  pj/crn-  or  less  with  cor¬ 
responding  interharmonic  distortions  and  hence  re¬ 
sponse  nonlinearities  below  1%  at  these  write  light 
energies.  The  sensitivity  of  the  device  defined  as  the 
write  light  exposure  Ew  necessary  to  achieve  t)  =  at 
5  cycles/mm  is  50  ^J/cm-.  In  our  CMU  data  on  these 
devices,7  8  an  available  laser  source  not  optimized  to  the 
thickness  of  the  Friz  used  was  employed.  With  this 
experimental  setup,  we  obtained  only  =  20  cycles/ 
mm  and  /,u  =  20-mm  resolution.  If  the  optimal  read 
light  wavelength  were  used,  we  expect  superior  results 
with  /,!->  in  excess  of  80  cvcles/mm  as  obtained  at 
FTI. 

Both  the  Friz  and  From  modulators  have  high  optical 
quality  and  allow  use  of  large  crystal  sizes.  The  Friz 
units  evaluated  at  CMU  had  a  15-mm  diam  active  area 
with  A/4  optical  quality.  Larger  units  have  been  fab¬ 
ricated  (up  to  20- mm  diam)  with  ~1  A  optical  flatness. 
The  standard  Priz  units  have  also  been  fabricated, 
tested,  and  used  with  A/10  optical  quality.  The  BSO 
crystal  in  the  Friz  is  nominally  400  pm  thick,  and  the 
insulating  layers  are  ~8  pm  thick.  No  substrate  is  used 
in  the  Friz,  or  the  Prom,  as  clamping  effects  change  the 
dielectric  constants  of  the  materials  and  induce  stresses 
in  the  device.  One  of  the  units  tested  at  CMU  is  shown 
in  Fig.  1. 

In  Table  1,  we  summarize  the  salient  Friz  perfor¬ 
mance  parameters.  These  data  were  obtained  from 
diverse  FTI  and  t’MU  tests  and  experiments.  As  with 
any  BSO  device,  a  wide  range  of  performance  is  possible 
depending  upon  the  thicknesses  used  for  the  different 
layers  in  the  device.  The  parameters  in  Table  I  are  all 
simultaneously  obtainable,  but  they  should  be  inter¬ 
preted  with  the  above  consideration  in  mind.  In  all 
cases,  the  device  should  be  operated  at  the  intended 
w'rite  light  exposures  E\y  and  write  light  wavelength  Au 
if  optimum  device  performance  is  to  be  obtained.  MTF 
data  are  frequently  used  to  describe  the  spatial  resolu¬ 
tion  of  SLMs.  Since  the  Priz  has  a  de  response  of  zero, 
the  MTF  function  r (/)  is  not  usable  for  such  a  device. 
Rather  the  diffraction  efficiency  y  of  such  a  device  is  the 


Table  I  Priz  Parameter  and  Performance  Specifications 


Parameter 

Specification 

Parameter 

Specification 

BSO  crystal 

100  pm  thick 

Address  lime 

7  nsec  (min! 

insulating  layer 

3  jim  thick 

Write/read  cvcle 

1  msec  ( milt) 

Klcctrndes 

I’t  or  InO  . 

Krase  time 

1  /usee  (mini 

Active  area 

Krase  cvcle 

1  msec  ttvpical) 

Typical 

Maximum 

13  mm  diam 

30  mm  diam 

Frame  rate  (typical! 

20  frames  sec 

Optical  quality 

Dark  Storage 

1  min  (typical) 

Typical 

Special  request 

,vt 

A  Jo 

Storage  with  readout 

*20  30 sec  i typical) 

Write  light 

Diffraction  effic.  ( rj,,* 

r. 

Wavelength 

330  300  nm 

Kesolut  ion 

Kxpnxurr  'typical) 

30  p.l/cnr 

at  i|  =  0.1 1|„ 

30  evdes  turn 

Sensitivity  (for  rj  =  lr<  at  3  cycles/mtn) 

30  pT'cm  ‘ 

at  rj  =  O.Oliio 

>  100  cycles  mm 

Head  light 

max.  measured 

>  300  evdes' mm 

,\  l typical) 

<>33  nm 

Spat  ial  freq.  response 

n  •  i  / 

Intensity  (typical) 

2  mW /cm  ’ 
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appropriate  parameter.  This  is  why  all  spatial  fre- 
quenev  resolution  data  in  Table  1  are  given  in  terms  of 
i)  vs  f.  These  data  were  obtained  with  the  Priz  device 
operated  dynamically  in  real  time.  In  coherent  optical 
processing,  the  amplitude  transmittance  mif  1  of  the 
Sl.M  vs  frequency  /  is  the  parameter  of  interest.  If  an 
analyzer  is  used  and  the  input  pattern  is  the  sine  wave 
grating  /  =  /,i(l  +  s\n'2nfx),  the  amplitude  transmit¬ 
tance  for  the  Friz  can  be  written  in  the  linear  approxi¬ 
mation  as 

tlx  )  =  mif)  rus^ff/.x  I  I) 

Equation  (4)  implies  that  the  do  component  of  the  Friz 
response  is  zero  [since  from  Eq.  (It)  A </>  =  0  at  /  =  ()|,  and 
the  readout  pattern  has  a  jt I'l  phase  shift  with  respect 
to  the  recorded  one.  In  this  case,  mif)  is  directly  ob¬ 
tained  from  the  r\if)  data  provided  by 

|m|  =  r /.  (5) 

This  expression  follows  from  the  Fourier  transform 
analysis  of  Eq.  (4). 

In  Fig.  2,  we  show  the  diffraction  efficiency  rj  of  Friz 
and  Prom  units  at  comparable  E\\-  =  nO-pJ/cm-  expo¬ 
sures.  From  these  data,  we  see  the  superior  diffraction 
efficiency  and  resolution  of  the  Priz.  We  also  see  that 
the  response  of  the  Friz  decreases  at  higher  spatial 
frequencies  at  a  much  lesser  rate  than  that  of  the  Prom. 
Where  mif)  vs  /  is  plotted  rather  than  ri(f),  the  differ¬ 
ences  would  he  even  larger.  We  also  note  that  the  Priz 
exhibits  a  suppressed  response  at  low  spatial  frequen¬ 
cies.  This  feature  follows  directly  from  the  fact  that, 
upon  uniform  illumination  of  the  device  with  write  light, 
a  uniform  longitudinal  electric  field  is  formed  that  has 
little  or  no  transverse  component.  Since  the  Priz  em¬ 
ploys  the  transverse  electrooptic  effect,  it  will  not 
modulate  in  response  to  such  light.  As  a  result  p  of  the 
Priz  device  peaks  at  a  spatial  frequency  of  4  cycles/mm 
(for  the  unit  tested)  rather  than  at  dc. 


IV.  Use  of  the  Priz  in  Pattern  Recognition  and  Signal 
Processing  Correlations 

An  attractive  optical  correlator  for  image  pattern 
recognition  is  the  joint  transform  correlator."’  In  this 
system,  the  reference  object  being  sought  is  placed  be¬ 
side  the  real-time  input  scene  in  the  input  plane  of  a  2-1) 
optical  FT  system.  The  objective  is  to  determine  if  the 
reference  object  is  present  in  the  input  scene  and  to 
determine  its  location.  Such  pattern  recognition  ap¬ 
plications  are  appropriate  for  locating  objects  on  an 
assembly  line  and  locating  areas  and  landmarks  in 
satellite  imagery  as  well  as  in  missile  guidance  and  many 
other  applications.  In  the  joint  transform  correlator, 
the  Fourier  transform  of  the  input  and  reference  data 
is  formed  on  an  intensity  sensitive  material  (such  as 
film,  the  Priz,  or  Prom).  The  Fourier  transform  of  this 
joint  FT  pattern  is  then  formed,  and  it  can  be  shown"' 
that  it  contains  the  correlation  of  the  input  and  refer¬ 
ence  images.  In  Fig.  2  we  show  an  example11  of  such  a 
correlation  performed  on  the  Priz.  The  FT  of  the  two 
input  objects  |Fig.  2(a)|.  identical  images  of  lobsters,  was 


1 
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Fig.  Uiffr.idion  efficiency  q  vs  spatial  frequency  f  lor  the*  From 
and  Priz  spatial  light  modulators. 


Fig.  Real  time  imago  pattern  recognition  with  the  Priz  used  in 
iho  Fourier  transform  plane  of  a  joint  Fourier  transform  correlator: 
fa)  input  and  reference  images:  thi  cross-sectional  scan:  and  it  l  an 
image  of  the  output  correlation  plane  pattern.  The  two  peaks  to  the 
right  and  left  in  tl>)  and  fc)  represent  tin*  correlations  of  the  two  input 
objects. ; 


recorded  on  a  Priz  placed  in  the  FT  plane  of  a  lens, 
which  was  behind  the  joint  input  pattern  of  Fig.  2(a). 
The  FT  of  the  data  recorded  on  the  Priz  is  shown  in  Fig. 
2(c)  and  its  cross-sectional  scan  in  Fig.  2(h).  This  full 
correlation  plane  pattern  contains  a  central  term  that 
is  the  sum  of  the  autocorrelations  of  each  input  object. 
The  large  spikes  on  the  left  and  right  in  Fig.  2(h)  are  the 
correlation  of  the  two  input  objects.  Their  presence 
indicates  that  the  two  input  objects  are  similar,  and  the 
relative  position  of  the  peaks  denotes  the  location  of  the 
reference  object  within  the  field  of  view  of  the  input 
image. 
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Signal  correlations  have  also  been  performed  on  the 
Priz.  An  image  of  a  linear  frequency  modulated  (I.F.M) 
signal  (Pig.  4 (a 1 1  was  recorded  on  (he  Priz.  The  FT  of 
t his  pattern  formed  with  a  lens  whose  focal  length  was 
matched  to  the  frequency  range  of  the  I.F.M  waveform 
yields  the  resultant  compressed  pulse  or  output  corre¬ 
lation  peak  ’  shown  in  Fig.  Ilbt.  The  width  of  the  ex 
per iment ally  obtained  correlal  ion  peak  was  1  lOUt  h  ot 
the  width  of  the  original  I.F.M  signal.  This  is  in  good 
agreement  with  the  theoretical  pulse  compression  factor 
of  120  for  the  I.F.M  signal  used.  The  difference  between 
theory  and  experiment  was  due  to  taper  in  the  input 
light  beam  and  other  effects. 

V.  Image  Spatial  Filtering  Using  the  Priz 

In  Fig.  2  and  Sec.  111.  we  noted  that  because  of  the 
transverse  electrooptic  effect  in  the  Priz.  dc  and  low 
spatial  frequency  data  are  automatically  suppressed. 
In  Fig.  atiil.  we  show  the  reconstruction  of  a  circular 
input  object  recorded  on  the  Priz.  As  expected,  only 
the  edge  contour  of  the  object  appears  (due  to  the  au¬ 
tomatic  dc  spatial  frequency  suppression  performed  by 
the  Priz).  The  amount  of  dc  suppression  depends  on 
how  closely  the  read  light  beam  is  incident  to  t  he  normal 
to  the  crystal.  In  our  experiments  at  (Ml',  we  were 
able  to  obtain  a  dc  suppression  of  Hi  when  t  he  normal 
to  Priz  was  aligned  within  lc  ot  the  read  beam.  For  a 
It)1  suppression  factor.  1°  alignment  is  necessary.  In 
the  image  processing  experiments  at  ('Ml' t Figs.  and 
7 ».  the  read  beam  was  incident  at  an  angle  of  1.7°  tot  he 
normal  to  the  crystal. 

The  Priz  has  anisotropic  properties  arising  Irotn  those 
of  the  linear  transverse  electrooptic  effect.1  In  par 
ticular.  it  exhibits  a  distinct  difference  in  response  to 
circularly  and  linearly  polarized  read  light.  Figure  ti 
shows  how  diffraction  efficiency  to  the  Priz  depends  on 
orientation  of  the  crystal's  axes  when  read  out  with 
linearly  and  circularly  polarized  light.  In  the  data  ol 
Figs,  a  and  7.  the  electric  vector  of  the  linearly  polarized 
light  was  along  the  (1  !2|  axis  of  t he  crystal.  In  Fig.  tl. 
the  diffraction  efficiency  >]  as  a  function  of  the  angle 
between  t  he  wave  vector  of  the  sine  wave  grating  and  t  he 
1 1  K>|  crvstal  axis  is  plotted  in  polar  coordinates. 

The  outer  circle  in  Fig.  b  describes  the  device's  re 
spouse  to  circularly  polarized  input  light.  As  seen,  it 
is  quite  uniform,  and  thus  operation  with  eircularlv 
polarized  input  light  produces  no  directional  preference 
for  input  spatial  frequency.  The  reconstructed  image 
in  Fig.  b(at  verifies  this  response  and  is  essentially  how 
the  circular  outer  curve  in  Fig.  ti  was  obtained,  How 
ever,  t  he  response  of  the  Priz  to  lincarlv  polarized  read 
light  is  quite  different.  In  the  two  inner  figure  eight 
shaped  curves  in  Fig.  ti.  we  show  the  response  for  I  in 
early  polarized  read  light.  When  the  dev  ice  is  exposed 
to  linearly  polarized  input  light .  it  exhibits  a  preferred 
response  >)  for  input  spatial  frequencies  oriented  in  one 
direction,  while  greatly  suppressing  input  spatial 
frequencies  oriented  in  the  ort  hogonai  direct  ion.  I'hc 
direction  in  which  spatial  frequencies  arc  suppressed 
can  be  controlled  by  the  polarity  of  t ho  voltage  applied 
to  the  modulator  if  the  polarizat  ion  of  t  he  read  light  is 
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Kij*.  DilTriitiion  t*t  1  it  it* tu\\  tj  lor  ihr  IVi/  as .» function  of  the  angle 
between  tlu-  wave  vector  ol  the  recorded  sine  wave  grating  and  the 
erystalVaxes  lor  input  read  light  with  circular  and  linear  jxilari/ation. 
t'urves  1  and  J  correspond  to  experimental  points  lor  linearly  polar 
i/ed  light.  and  curve  corresponds  to  circularly  polarized  read  light. 


fixed.  The  two  figure  eight  shaped  plots  in  Fig.  ti  were 
obtained  with  the  same  polarity  of  the  read  light  but 
with  different  polarities  of  the  applied  voltage.  Dif¬ 
ferences  arising  from  changing  the  voltage  polarity  can 
!>e  attributed  to  the  optical  activity  of  the  BSO  crystal.17 
Figure  5(b)  shows  the  reconstructed  image  of  a  circular 
object  recorded  on  the  Friz  and  read  with  linearly 
polarized  light.  As  seen,  the  spatial  frequencies  in  one 
direction  are  suppressed  as  predicted  by  Fig.  t>. 

For  i  [1 10)  cut  Friz,  similar  plots  of  >j  vs  the  read  wave 
vector's  direction  result.  However,  for  circularly 
polarized  read  light,  a  saddle-shaped  response  rather 
than  a  circular  one  results.  Similarly,  a  larger  ij  (a  factor 
of  2  larger  than  for  the  ( 1 1 1 1  cut  Friz.)  results  when  the 
1 1 10|  cut  device  is  operated  with  linearly  polarized  read 
light.  Thus  the  1 1  ll)|  cut  device  is  preferable  for  mul¬ 
tichannel  I  T)  signal  processing  applications  and  others 
in  which  directional  spatial  filtering  is  desired.  Con¬ 
versely  the  |1 1 1 1  cut  Friz  is  preferable  for  image  pro¬ 
cessing  where  a  uniform  response  is  generally  desired 
for  all  input  spatial  frequency  directions. 

To  achieve  a  high  degree  of  suppression  of  the  dc 
component  in  the  image  (both  with  linearly  and  circu¬ 
larly  polarized  read  light),  the  modulator  should  be 
placed  between  a  high-quality  polarizer  and  analyzer, 
which  in  the  case  of  circularly  polarized  light  can  be 
achieved  with  a  A/4  wave  plate  and  a  linear  polarizer. 
The  dc  suppression,  directional  spatial  filtering,  and 
edge  enhancement  features  of  the  Friz  are  quite  useful 


preprocessing  operations  for  multisensor  and  IK  pattern 
recognition.  In  Fig.  7.  we  show  the  original  IK  image 
|Fig.  7(a)).  the  image  constructed  from  a  1 1 10)  cut  Friz 
with  the  read  light  polarized  at  45°  |Fig.  7(b)|  and  with 
vertically  polarized  read  light  |Fig.  7(c)).  The  recon¬ 
structed  image  in  Fig.  7(b)  approximates  an  edge-en¬ 
hanced  version  of  the  original  image,  whereas  the  re¬ 
construction  in  Fig.  7(c)  results  in  enhancement  of 
vertical  lines  in  the  original  image  and  suppression  of 
horizontal  spatial  frequencies  in  the  original  input 
pattern. 

VI.  Dynamic  Image  Selection 

In  investigating  the  response  of  the  Friz  to  spatially 
moving  2-1)  input  patterns,  it  was  found1  that  the  de¬ 
vice’s  response  was  a  function  of  both  the  spatial  fre¬ 
quency  of  the  input  data  and  the  velocity  with  which  the 
input  data  moved  across  the  input  field  of  view.  A 
modified  version  of  the  Friz  was  used  in  these  experi¬ 
ments.  It  had  no  insulating  layers,  so  electrodes  were 
evaporated  directly  on  the  crystal’s  surface. Is  The  re¬ 
sponse  of  the  device  to  an  0.5-mm  wide  input  line  was 
measured  for  different  velocities  ( 1  40  mm/sec)  of  the 
input  object  across  the  input  plane,  and  it  was  found1 
that  the  response  of  the  device  peaked  when  the  velocity 
was  ~7  mm/sec.  The  response  of  the  Friz  is  thus  a 
function  of  both  time  and  space  ( i . e . .  the  spatial  fre¬ 
quency  of  (he  input  data  and  the  rate  at  which  it 
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Ki>i.  S.  St'hfinalir  diagram  <>t  tin-  opiual  systrin  iistsl  to  drmniistratr 
dynamic-  image  selection  i>r  change  detection  « >n  the  I'ri/. 


changes  or  moves  with  time).  An  initial  attempt  to 
describe  the  combined  time  and  space  response  of  the 
device  has  been  reported.  ’1"  Initial  experiments'1" 
indicate  that  as  the  spatial  frequency  of  the  input  data 
decreases,  the  input  velocity  for  which  the  response  >1 
of  the  device  peaks  shifts  to  higher  velocities. 

To  obtain  t his  effect,  the  I’ri/  is  operated  with  a  fixed 
voltage  across  its  electrodes  rather  than  with  the  applied 
voltage  switched  between  positive  and  negative  polar¬ 
ities.  In  the  normal  mode  (polarization  of  the  applied 
voltage  switched),  the  device  performs  like  the  From. 
Only  with  a  fixed  voltage  will  it  perform  dynamic  image 
selection.  In  this  mode,  the  device  responds  only  to 
changes  in  the  input  image.  This  operating  mode  is 
attractive  for  many  applications  such  as  change  detec¬ 
tion,  and  it  also  greatly  simplifies  the  electronic  support 
system  necessary  (since  a  fixed  rather  than  a  switching 
high-voltage  supply  can  he  used).  Over  a  selected  range 
of  input  temporal  frequencies  /.  (where  this  range  varies 
as  a  function  of  the  input  spatial  I'requetuy  /, ).  the  >}  vs 
/,  response  is  linear,  and  the  device  performs  a  time 
differentiation  of  the  input  data.  This  range  of/,  is 
quite  small,  and  moreover  it  varies  with  the  intensity 
of  the  input  write  light.  For  these  reasons,  this  Friz 
device  features  is  best  termed  dynamic  image  selection 
(i.e..  the  device's  output  represents  only  the  changing 
part  of  the  input  data)  rather  than  temporal  differen¬ 
tiation.  1  This  effect  can  also  he  observed  with  a  fixed 
input  and  with  the  write  light  beam  pulsed  on  and  off. 
In  this  case,  whenever  the  write  light  changes  (goes  on 
or  off),  an  output  image  of  the  input  data  appears  and 
then  decays  with  a  time  constant  that  is  a  function  of  the 
intensity  of  the  write  light.  If  the  differential  phase  Ao 
of  the  output  light  is  measured,  it  is  seen  to  be  of  op¬ 
posite  sign  when  the  write  light  is  switched  from  off  to 
on  compared  to  when  it  is  switched  from  on  to  off. 

When  the  Friz  units  were  being  tested  at  CMC.  we 
found  this  Friz  feature  to  be  most  attractive  and  thus 
assembled  the  system  of  Fig.  8  to  demonstrate  the  use 
of  the  Friz  in  change  detection.  The  system  of  Fig.  8 
contains  two  input  planes.  Flane  l\<  contained  a  fixed 
image,  in  our  case  a  random  pattern  of  uncorrelated 
noise  and  correlated  noise  of  different  correlation 
lengths  ami  with  different  mean  values.  This  fixed  /*,. 
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|  >a  I  tern  was  imag'd  onto  I  he  I  ’ri/  at  l’\  thy  i mao  ini:  If  ns 
II  1  together  wit  li  t  lie  I'" I'  <>!'  an  acotistonpticl  t  A(  )i  cell 
l Usin*:  tin1  !•'  1’  lens  /•  / 1. 1.  I'lu-  At )  fell  was  operated  in 
the  scannini:  mode  with  a  repealed  I.FM  input  signal, 
flits  eattsed  a  scannini:  spot  to  traverse  the  1  ’ri/  (sit 
perintposed  on  the  fixed  noise  pattern  from  thus 
simulating  a  moving  object  on  a  noise  haek^roimd.  The 
veloeilv  of  the  scannini:  spot  was  adjusted  to  he  AN 
mm  see.  and  its  si/e  was  ^1  pixel  (10  amt. 

Both  the  fixed  and  moving  input  patterns  were  im 
aned  onto  the  I’ri/.  in  Xu  =  ITb-nni  liftlit  from  an 
arpoti  ion  laser.  Readout  was  performed  in  A/,-  = 
b:t:i  mil  lirtht  incident  normal  tot  he  I’ri/  as  shown  in  t  lie 
left  side  of  Fiu.  N.  The  pattern  on  the  I’ri/  was  then 
imaged  onto  /’  ustni;  imaoini:  lens  II.  through  a  crossed 
analyzer  and  a  fvia-nm  filter  t/V/.l  The  output  /’ 
pattern  was  delected  on  a  vidieoti.  and  the  dynamic 
moving  output  was  visible  on  an  isometric  display  where 


it  could  lie  photographed.  A  filler  optic  iFtli  probe 
with  a  microscope  and  I’M T  was  also  plated  at  /'  !>• 
allow  (plant  i  I  at  ive  measurements  ot  tin  output  plant 
to  he  made. 

In  F  iy.  tit  at.  we  show  t  he  I  nil  out  put  imam  at  /  ’  (with 
the  hifdi  voltage  I’ri/  power  'tipplv  operated  in  tin 
normal  pulsed  model.  This  output  'hows  the  lived 
background  noise  pattern  and  the  time  mu  »pot.  (Tin 
moving  object  or  spot  i'  present  in  I  lie  back  let  t  ot  the 
figure. >  When  the  Inch  vollaste  l’n/ power  stipplx  po 
lari t y  was  fixed,  only  the  time  \ar\ini;  portion  ot  the 
input  pattern  appeared  at  I  he  output.  In  I  Ills  case,  on l\ 
t  he  linn  ■  nil  spot  produced  by  t  he  scanning  A(  I  cell  wa~ 
v  isible.  In  Fi.u- bl.  we  show  t he /’  output  tor  one  lo 
cation  of  the  scanning  spot  ti orrespotidmn  to  a  aimi 
lated  moving  target  in  the  constant  noise  ba<  kjtroiind 
Ullage  I.  Asian  lie  seen.  I  he  I’ri/  suppresses  the  t  ixed 
background  noise  quite  well. 


t  November  15181'  Vol  .' 1  No  .’1  AITHItr  OPTICS 


I8S.1 


Table  II  Quantitative  Dynamic  Image  Selection  Data  on  the  Priz 

A\„  I  in  jr. 

U .  .'i  '>0  is  lit 


To  quantify  the  amount  of  background  suppression 
obtained,  the  intensity  of  the  moving  spot  was  varied 
and  the  ratio 

fl  intensitv  of  tht*  moving  spot 

f\  =  - - - ; -  Mil 

average  intensity  of  I  he  fixed  background 

was  measured  at  the  input  and  output  of  the  system 
(i.e..  with  and  without  dynamic  image  selection  or 
change  detection).  The  results  are  summarized  in 
Table  II.  From  these  data,  we  note  that  once  the  in¬ 
tensity  of  the  scanning  spot  has  been  increased  so  that 
/f,„  =  2.5  or  greater,  a  constant  /f,„„  ~  50  ratio  results 
with  the  intensity  of  the  dynamic  part  of  the  output 
image  being  50  times  the  average  background  level  in 
the  /'.  output.  This  occurred  because  when  Rm  was 
increased  above  2.5.  saturation  of  the  Friz  occurred,  and 
thus  no  further  changes  in  the  effective  Rm  resulted. 
This  is  expected  since,  with  the  Priz  in  the  FT  plane  of 
the  AO  cell,  all  the  light  from  the  cell  was  concentrated 
onto  a  single  pixel  on  the  Priz. 

VII.  Summary  and  Conclusion 

A  complete  and  unifying  summary  (at  this  present 
time)  of  a  new  light  modulator,  the  Priz,  has  been  pre¬ 
sented.  Many  new  experimental  demonstrations  re¬ 
cently  obtained  at  (’Ml1  were  included.  The  theory  of 
operation  of  the  Priz  was  reviewed  and  experimentally 
verified,  and  its  similarity  and  differences  from  the 
Prom  were  noted  (Sec.  II).  The  first  unifying  summary 
of  the  performance  parameters  of  the  Priz  was  also  ad¬ 
vanced  (Sec.  III).  A  summary  of  many  of  the  possible 
applications  of  the  Priz  was  then  presented.  This  in¬ 
cluded  conventional  optical  pattern  recognition  and 
optical  signal  processing  correlators  (Sec.  IV)  plus  three 
new  image  processing  operations:  dc  suppression  and 
directional  filtering  (Sec.  V)  plus  dynamic  image  se¬ 
lection  (Sec.  VI).  'The  first  image  processing  operation 
had  been  experimentally  demonstrated  previously. 
Directional  spatial  filtering  demonstrations  had  not 
been  previously  described.  Our  dynamic  image  selec¬ 
tion  experiments  together  with  Ref.  18  represented  the 
first  examples  of  the  use  of  the  device  for  the  selection 
of  the  dynamic  part  of  an  image  from  a  constant  fixed 
noise  background  (i.e..  change  detection). 


Many  aspects  of  the  Priz  device  are  well  understood. 
However,  further  theoretical  analyses  and  modeling 
together  with  further  device  fabrication  and  experi¬ 
mental  testing  are  necessary  to  understand  fully  and 
describe  many  of  the  observed  features  of  the  device. 
Issues  meriting  further  analysis  include  the  nonlinearity 
associated  with  the  transverse  electrooptic  effect,  the 
combined  time  ami  space  dependence  of  the  resolution 
of  the  device,  and  a  theoretical  formulation  of  the  dy¬ 
namic  image  selection  feature  of  the  device  with  at¬ 
tention  to  the  selection  of  device  parameters  to  optimize 
and  control  this  effect. 

The  promising  performance  parameters  tabulated  for 
the  Priz  light  modulator  and  the  experimental  verifi¬ 
cation  included  of  several  of  the  novel  features  of  this 
device  indicate  that  a  wealth  of  new  research  efforts  and 
applications  are  still  possible  in  the  field  of  real  time  and 
reusable  spatial  light  modulators. 
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ABSTRACT 

The  basic  acousto-optic  signal  processing  architectures  (spectrum  analyzer,  space-inte¬ 
grating,  time-integrating  and  triple  product  processor)  systems  and  algorithms  such  as  the 
chirp-Z  transform  are  reviewed.  We  then  describe  new  acousto-optic  data  processing  systems 
and  applications  that  utilize  these  basic  architectures  and  new  ones.  These  include  a 
matched  spatial  filter  acousto-optic  processor,  two  new  hybrid  time  and  space-integratir.g 
systems,  a  triple  product  processor  and  four  new  matrix-vector  iterative  feedback  systems. 

1.  INTRODUCTION 

Acousto-optic  (AO)  transducers  are  commercially  available  real-time  spatial  light  modu¬ 
lators  that  can  be  used  for  many  applications  and  system  architectures.  Such  devices  ar.c 
systems  have  received  considerable  attention  (1).  In  Section  2,  we  review  several  of  the 
more  basic  AO  system  architectures.  In  Section  3,  we  discuss  a  multi-channel  AO  processor 
[2l  architecture  in  which  multiple  signal  channels  are  accommodated  by  multiple  LEDs  or 
laser  diodes  (LDs)  rather  than  by  a  mul ti -channel  AO  cell.  One  of  the  major  attractive 
reasons  for  using  AO  processors  is  the  large  processing  gain  (PG)  and  time  bandwidth  product 
(TBWP)  possible  on  such  systems.  In  Section  4,  we  discuss  PG,  define  various  possible  out¬ 
put  correlation  plane  SNR  measures  and  provide  experimental  verification  of  our  remarks  .3;. 
We  also  discuss  a  technique  to  facilitate  generation  of  the  Iona  codes  for  which  AO  systems 
are  useful  [4,5).  In  Section  5,  we  describe  several  new  AO  processors:  a  matched  spatial 
filter  (MSP)  AO  correlator  that  reduces  the  effects  of  non-uniform  spatial  AO  cell  response 
variations  [3),  two  new  hybrid  time  and  space-integrating  (TSI)  architectures  that  simul¬ 
taneously  achieve  the  best  features  of  a  space-integrating  and  time-integrating  processor 
(4],  and  a  new  application  f 5 1  of  the  triple  product  processor.  In  Section  6,  we  address  a 
major  new  application  area  for  AO  devices:  optical  systolic  array  matrix-vector  and  itera¬ 
tive  optical  processors.  We  describe  the  basic  optical  matrix-vector  feedback  system  ar.d 
note  the  general-purpose  nature  of  such  a  processor.  We  then  discuss  two  deconvolution 
architectures  using  such  a  concept,  an  optical  systolic  matrix-vector  multiplier  and  an  op¬ 
tical  systolic  system  for  solving  matrix-vector  equations  (6).  We  then  conclude  with  a  new 
frequency-multiplexed  version  of  these  systems  |7)  and  our  summary  and  conclusions  (Section 
7)  . 

2.  BASIC  ACOUSTO-OPTIC  SIGNAL  PROCESSING  ARCHITECTURES 

Various  reviews  exist  of  the  possibilities  of  optical  signal  processing  systems  and  arch¬ 
itectures  (1,8  —  10)  .  Thus,  this  present  discussion  of  the  same  subject  will  be  quite  brief. 
First,  we  note  that  an  AO  cell  illuminated  with  parallel  laser  light  diffracts  this  light  at 
angles  proportional  to  the  frequencies  present  in  the  input  signai  and  with  the  amplitude  of 
each  diffracted  wave  proportional  to  the  strength  of  each  input  signal  component.  Thus,  a 
lens  placed  behind  an  AO  cell  forms  the  Fourier  spectrum  of  the  input  signal  data  in  its 
back  focal  plane.  This  spectrum  analysis  feature  of  AO  devices  is  presently  being  used  in 
many  applications  (1).  One  particularly  attractive  application  uses  an  AO  cell  with  two 
transducers  or  two  separate  AO  spectrum  analyzers,  each  fed  with  a  signal  from  a  different 
antenna  pointed  in  a  different  direction.  From  the  location  of  peaks  in  the  two  resultant 
output  Fourier  transform  (FT)  planes,  the  frequency  distribution  of  emitters  present  in  the 
f ield-of-view  of  the  antennas  can  be  obtained.  From  the  ratio  of  the  amplitudes  of  the  out¬ 
put  peaks  on  detectors  corresponding  to  the  same  frequency  component,  estimates  of  the  di¬ 
rection  of  each  emitter  can  be  obtained.  The  resultant  output  information  from  such  a  sys¬ 
tem  can  thus  be  used  to  obtain  a  2-D  (f requency/direction-of-arrival ,  or  f/DOA)  display  of 
the  emitters  present  in  the  f ield-of-view  of  the  antennas  (11).  Such  displays  are  most  use¬ 
ful  for  many  signal  processing  applications. 

The  space-integrating  (SI)  AO  correlator  or  convolver  system  [12]  of  Figure  1  is  a  basic 
AO  processor  architecture.  In  this  system,  the  signal  g(t)  is  fed  to  an  AO  cell  at  plane 
P1A'  whose  transmittance  is  a  function  of  time  t  and  space  x  given  by  g(t-x/vs)  =  g(t-x')  or 
g(x'-t),  where  vs  is  the  velocity  of  sound  in  the  AO  cell.  This  pattern  is  then  imaged  onto 
a  mask  h(x')  at  plane  Pig.  The  light  distribution  g(t-x’)h(x’)  or  g(x'-t)h(x’)  is  then 
spatially  integrated  by  lens  Lj  and  the  time-history  output  from  the  photodetector  at  P3  is 
the  correlation 


h  ®  g  =  /g(x*  -  t)h(x')dx’  =  R(t) 
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or  the  convolution 


h  *  g  =  / g(t  -  x'lhlx'ldx'  =  C(t)  (It, 

of  the  two  signals  g  and  h.  In  this  archi tecture ,  the  integration  in  (1)  is  performed  over 
space  and  the  convolution  or  correlation  output  is  displayed  with  the  shift  variable  t  be i r.g 
time  t  as  in  (1).  We  thus  refer  to  this  architecture  as  a  space- i ntearati no  convolver  or 
correlator . 
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FIGURE  1  Schematic  diagram  of  the  basic  space  FIGURE  2  Schematic  diagram  of  the  basic 
integrating  acousto-optic  convolver  or  corre-  time-integrating  acousto-optic  convolver 
lator.  or  correlator. 


The  third  basic  AO  processor  architecture  is  the  time-integrating  (TI )  convolver  or  cor¬ 
relator  [13]  of  Figure  2.  In  this  system,  the  signal  g(t)  is  used  to  time-sequentially  mod¬ 
ulate  the  output  from  an  LED,  LD  or  point  AO  cell.  This  output  uniformly  illuminates  an  AC 
cell  at  P3  fed  with  a  signal  h(t).  With  the  transmittance  of  this  AO  cell  described  by 
h(x'-t)  or  h(t-x'),  the  light  distribution  leavina  P3  is  the  product  of  g ( t )  and  the  trans¬ 
mittance  of  the  AO  cell.  The  AO  cell  is  then  imaged  onto  a  linear  detector  array  at  P3, 
where  time-integration  of  the  product  is  performed  directly  on  the  detector.  The  resultant 
output  at  P3  for  this  system  is  again  either  the  correlation 

g  ®  h  =  /g(t)h(t  -  x'Jdt  =  R(t)  (2a 


or  the  convolution 


g  *  h  =  /q  ( t )  h  (x '  -  t)  dt  =  C  (  t  )  .  (2b! 

Since  the  integration  in  (2)  is  performed  in  time,  we  refer  to  this  as  a  time-intearatir.g 
convolver  or  correlator  and  we  note  that  the  shift  variable  t  in  the  output  correlation  or 
convolution  plane  P3  is  the  space  variable  x' . 

The  space-integrating  system  of  Figure  1  enjoys  a  large  ranae  delay  search,  but  can  only 
operate  on  signals  whose  TBWP  equals  that  of  the  AO  cell  (typically  1000  to  2000) .  Converse¬ 
ly,  the  TI  system  of  Figure  2  can  only  search  a  range  delay  equal  to  the  aperture  time  or 
transit  time  Tft  of  the  AO  cell  (typically  l-50usec) ,  but  can  process  a  signal  of  long  dura¬ 
tion  (>>  T ft)  and  large  TBWP  and  hence  can  provide  a  large  PG  and  integration  time  Tj  (limit¬ 
ed  by  the  dynamic  range  of  the  detector  system  used) . 

The  fourth  basic  AO  architecture  is  the  triple  product  processor  (TPP)  system  [14]  (see 
Figure  7  in  Section  5).  In  this  system,  a  point  light  source  uniformly  illuminates  one  AO 
cell  (oriented  horizontally) .  The  light  distribution  leaving  this  AO  cell  is  compressed 
horizontally  and  expanded  vertically  to  illuminate  a  second  AO  cell  (oriented  vertically). 
Both  AO  cells  are  then  imaged  onto  an  output  plane  P3  where  time-integration  occurs  on  a  2-D 
detector  array.  If  the  signals  to  the  point  modulator  and  the  two  AO  cells  are  denoted  by 
u(t),  v(t)  and  s(t),  then  the  2-D  output  at  plane  P3  is  the  triple  product 

R(t2»ti)  =T/U(t)v(t  -  Tilsit  +  T2>dt,  (3) 


0' 


Vjc- 


and  hence  the  name  TPP  has  been  given  to  such  an  architecture.  This  system  has  considerable 
versatility,  since  its  2-D  output  plane  pattern  can  be  a  folded  spectrum,  an  ambiguity  sur¬ 
face  or  (as  we  show  in  Section  5)  a  coarse/fine  time  delay  display. 

One  particular  algorithm  that  has  provided  each  of  these  latter  three  AO  architectures 
with  an  increased  flexibility  is  the  chirp-Z  transform  (CZT)  algorithm  I  15).  We  describe 
this  algorithm  with  respect  to  the  TI  system  of  Figure  2.  If  the  input  signal  g(t)  is  a 
signal  f(t)  multiplied  by  a  chirp  or  LFM  waveform  and  if  the  input  signal  h(t)  to  the  AO 
cell  is  also  a  chirp  and  if  the  opposite  sidebands  of  each  chirp  are  filtered  within  the 
optical  portion  of  the  system,  then  the  1-D  output  pattern  at  P3  can  be  shown  to  be  the 
Fourier  transform  (FT)  of  the  input  signal  f(t).  This  same  technique  can  be  applied  to  the 
TPP  system  to  yield  a  2-D  FT  output  with  coarse  and  fine  frequency  axes  (i.e.,  a  folded 
spectrum  output  display) . 


3.  MULTI-CHANNEL  ACOUSTO-OPTIC  PROCESSING 


There  are  various  ways  to  achieve  multi-channel  and  2-D  processing  usino  1-D  AO  devices. 
The  use  of  crossed  AO  cells  (as  in  the  TPP  system)  is  one  technique.  The  use  of  AO  cells 
with  separate  signal  channels  on  the  same  cell  is  a  second  approach.  In  Figure  3,  we  show 
an  alternate  architecture  (2)  in  which  a  single  channel  AO  cell  and  a  linear  array  of  LEDs 
or  LDs  are  used  to  achieve  a  multi-channel  system.  The  input  signal  to  the  AO  cell  in  this 

system  of  Figure  3  is  a(t)  and  the  transmittance  of  the  cell  is  uniform  along  any  vertical 

line.  The  input  signals  to  the  N  LEDs  are  f^  to  f(j.  The  output  pattern  on  the  N  1-D  detec¬ 
tors  is  then  the  N  correlations  fn  ®  g.  Such  an  architecture  thus  realizes  N  1-D  correla¬ 
tions  of  g(t)  with  the  N  input  signals  fn(t).  If  the  CZT  algorithm  is  used,  the  output  pat¬ 
tern  contains  the  N  1-D  Fourier  transforms  Fn(u)  of  the  N  input  siqnals  fn(t).  Such  an 
architecture  is  useful  for  imaging  FT  spectroscopy  12].  If  the  N  input  signals  originate 

from  N  antennas  pointed  in  N  different  directions,  the  output  can  also  be  an  f/DOA  display. 

This  architecture  is  thus  very  attractive,  since  the  same  basic  system  can  be  used  for  mul¬ 
tiple  different  applications  (such  as  the  ones  mentioned  above)  simply  by  controllma  the 
input  electronic  signals  used. 


AO 


FIGUPE  3  Simplified  schematic  diagram  of  a  multi-channel 
acousto-optic  general-purpose  signal  processor 
(2)  . 


4 .  PROCESSING  GAIN  AND  SNR  CONCEPTS 


One  attractive  feature  of  optical  signal  processors  is  their  ability  to  accommodate  sig¬ 
nals  of  large  duration  and  long  Tj  and  large  TBWP.  The  TI  architectures  are  especially  at¬ 
tractive  for  realizing  such  features.  In  such  cases,  a  most  attractive  performance  measure 
is  PG.  This  is  defined  as  the  ratio  SNR0/SNR^  of  SNRq  at  the  output  of  the  processor  to  the 
SNRj_  at  the  signal  input  to  the  system.  In  many  cases,  the  signal  code  used  is  best  de¬ 
scribed  statistically,  the  signal  is  then  a  sample  realization  of  a  random  process  and  we 
refer  to  the  statistical  correlation  or  the  ensemble  average  (denoted  by  E)  of  two  signals 
S3 (t)  and  S2 (t) ,  i.e. 


R(x)  «  E{s3<t)s2<t  +  t)K 


(4) 


We  have  highlighted  the  PG  and  three  different  SNRg  measures  for  optical  systems  in  (3). 
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Let  us  first  discuss  the  three  different  SNR0  measures  possible.  The  classical  SNP0 
measure  used  is  the  ratio  of  the  average  peak  intensity  of  the  output  correlation  divided 
by  the  variance  at  the  peak  (assumed  to  lie  at  t  =  t^  =  0  for  simplicity)  averaged  over  many 
realizations  of  the  process 

SNR1  =  E2(C  (0)  '/Var'C  (0)  !• .  (5) 


We  refer  to  this  as  SNRq  at  the  peak.  A  second  useful  SNRQ  measure  is  the  peak-to-sidelobe 
ratio  (PSR)  or  the  value  of  the  peak  intensity  divided  by  the  variance  of  the  correlation 
far  from  the  peak 


SNR2  '  VarTclfT" 


The  SNR0  measure  in  (5)  is  difficult  to  measure  experimentally  since  it  requires  evaluation 
at  one  point  and  thus  requires  many  repetitions  of  the  experiment.  The  SNRC  measure  m  (6) 
is  easily  obtained  from  one  experiment.  However,  the  dependence  of  this  measurement  or.  the 
size  and  location  of  the  r  >>  0  region  used  to  estimate  the  variance  in  (6)  car  affect  this 
SNRC  measure.  Thus,  a  new  SNRQ  measure 


SNR3 


_  e2'.c(0)  :• _ 

T  /2 

(l'T_)/  Var(Ci  ,  jdr 


(7) 


was  defined,  in  which  the  variance  is  obtained  for  the  full  T^  =  T^  range  of  output  delays 
being  searched. 

From  an  optical  experimental  standpoint,  measurement  of  SNRj  =  SNRq  is  (6)  is  preferable. 
In  (31,  we  showed  that  for  the  case  of  PRN  signal  with  high  noise  (low  SNR^  that  SNR}  = 

SNR2  and  that  for  low  noise  (high  SNR^,  SNR}  =  *  and  SNR2  =  TBWP.  In  the  practical  low 
noise  case,  the  simpler  SNR,  measure  thus  yields  the  same  results  as  the  more  difficult  SNR} 
measure  of  SNR0.  However,  for  the  case  of  passive  signals  in  which  the  recieved  signal 
amplitude  is  continuous  rather  than  discrete,  the  signal  is  well-modeled  as  Gaussian  and  for 
this  case  we  showed  (3)  that  the  numerators  in  (3)  and  (6)  will  be  identical  and  that  for 
such  Gaussian  signals  SNR}  =  SNR2  -  3dB  (if  SNR.^  =  a,  i.e.  no  noise).  For  the  case  of  Gaus¬ 
sian  signals  and  SNR}  =  0  (equal  amounts  of  signal  and  noise),  we  find  SNR2  -  SNR}.  We  con¬ 
clude  by  noting  that  SNR}  is  the  more  appropriate  measure  for  communications  (when  the  pro¬ 
cessor  is  in  synchronization  and  when  location  or  position  of  the  correlation  peak  is  known). 
Conversely,  SNR2  is  more  appropriate  for  parameter  estimation  (such  as  when  the  location  or 
time  delay  of  the  correlation  peak  is  desired) . 

In  [5],  we  discussed  how  the  bandwidth  of  the  noise  with  respect  to  the  bandwidth  of  the 
signal  code  affected  PG  and  we  noted  that  for  narrowband  noise  PG  =  TBWP  (of  the  signal)  or 
TSBS  and  for  wideband  noise  PG  =  ITBWP  (of  the  system)  or  TjBs.  This  distinction  is  impor¬ 
tant  when  the  code  used  is  repeated  and  when  integration  is  performed  over  repeated  versions 

of  the  code  (Tj  >>  Ts) .  As  we  have  shown  (3,5)  in  such  cases,  the  PG  obtained  will  be  less 

(i.e.  PG  =  TBWP)  if  the  noise  is  narrowband  than  if  it  is  wideband  (in  this  case  PG  =  ITBWP 
>>  TBWP) .  To  demonstrate  that  PG  =  TBWP  rather  than  ITBWP  for  the  case  of  narrowband  noise 
and  to  demonstrate  the  use  of  our  SNR2  measure  of  SNR0,  we  (3)  used  a  signal  with  a  TBWP  = 

2047.  We  repeated  this  signal  approximately  150  times  and  integrated  the  output  on  the  TI 

system  of  Figure  2  for  the  full  150  cycles  of  the  signal  code.  The  TBWP  =  2047  corresponds 
to  a  PG  =  33dB  and  the  ITBWP  of  approximately  300,000  corresponded  to  a  PG  =  55dB. 

In  Figure  4,  we  show  the  results  of  correlations  of  this  signal  with  narrowband  jammers 
of  different  strengths  (different  SNR}  levels).  The  results  obtained  verified  that  the  PG 
of  such  a  system  is  TBWP  not  ITBWP  and  it  also  verified  that  our  SNR2  measure  gave  adequate 
results  that  were  more  easily  obtained  in  the  laboratory.  These  experiments  and  our  associ¬ 
ated  theory  show  the  need  for  long  codes  and  processors  capable  of  coherent  integration  of 
codes  of  long  duration  and  large  TBWP.  Many  applications  require  such  performance  tooether 
with  the  need  to  search  large  range  delays  between  the  received  and  transmitted  signals.  In 
Section  5,  we  discuss  several  such  processors  and  we  also  discuss  ways  to  improve  the  uni¬ 
formity  of  the  correlation  plane  outputs  shown  in  Figure  4. 

A  final  important  and  practical  aspect  associated  with  the  realization  of  a  correlation 
of  a  code  with  a  very  large  TBWP  is  the  code  generator  itself.  A  very  attractive  technique 
for  generation  of  a  code  with  a  large  TBWP  is  to  produce  two  codes  0  and  0  with  Nu  and  Nv 
bits  respectively  and  to  repeat  each  code  Nv  and  Nu  times  respectively  in  tvo  seoarate  signal 
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generators.  If  we  then  form  the  bit-by-bit  product  of  each  code  (Figure  5),  we  obtain  a 
product  code  with  a  period  equal  to  NgTp  “  NuNvTg  or  fhe  product  of  the  durations  of  each 
code  (where  TB  is  the  duration  of  one  bit  of  the  code) . 


(a)  SNRj  -  +»  (b)  SNRj  -  -20dB  (c)  SMR^  -  -25dB 


FIGURE  4  Photographs  of  several  correlation  plane  output  patterns  from  the  time- 
integrating  correlator  of  Figure  2  demonstrating  the  processing  gain  and 
SNR  measurement  techniques.  The  signal  used  was  a  pseudorandom  code  with 
various  narrowband  noise  jammers  present  I  3) . 


FIGURE  5  Simplified  representation  of  the  generation  of  a  product  code  |4). 


5.  NEW  ACOUSTO-OPTIC  ARCHITECTURES  AND  APPLICATIONS 

In  this  section,  we  briefly  discuss  four  new  AO  architectures  and  applications.  First, 
we  recall  the  conventional  MSF  optical  correlator  [16]  redrawn  as  in  Figure  6.  If  we  place 
an  AO  cell  in  the  input  plane  and  an  MSF  of  the  reference  signal  at  plane  P2,  and  in  the 
output  use  a  parallel  output  shift  and  summation  detector  system  (detailed  below) ,  we  obtain 
a  new  hybrid  time  and  space-integrating  (TSI)  architecture  that  we  refer  to  as  an  AO  MSF 
correlator  (3).  Its  operation,  performance,  architecture  and  features  differ  considerably 
from  thoseof  the  prior  AO  systems.  First  recall  that  an  MSF  correlator  is  shift-invariant, 
i.e.  the  location  of  the  output  correlation  peak  corresponds  to  the  location  of  the  signal 
in  the  input  plane.  If  an  AO  cell  is  placed  at  the  inDut  plane  of  an  MSF  correlator  (as 
shown  in  Figure  6),  then  the  output  correlation  peak  will  move  across  the  output  linear  de¬ 
tector  array  as  the  input  signal  traverses  the  AO  cell.  At  each  detector  element  at  each 
time  instant,  we  have  a  partial  correlation  with  the  full  PG  of  the  TBWP  of  the  AO  cell.  We 
detect  one  such  correlation  output  pattern  and  the  output  pattern  at  the  next  time  instant 
(the  correlation  peak  will  now  lie  on  an  adjacent  detector) .  We  delay  these  two  detector 
outputs  by  one  bit  time,  add  them  and  repeat  the  process  for  N  bit  times.  We  then  obtain 
the  noncoherent  summation  of  the  N  partial  correlation  outputs,  each  with  a  PG  =  TBWP  of  the 
AO  cell.  Thus,  such  a  system  yields  a  large  PG.  But  also,  since  each  correlation  output  is 
obtained  for  a  different  location  of  the  signal  in  the  AO  cell,  the  final  correlation  output 
has  the  effects  of  spatial  non-uniformities  in  the  AO  cell  removed  (or  integrated  over  the 
aperture  of  the  cell).  We  have  demonstrated  the  use  of  such  a  system  [3]  for  an  AO  cell 
with  large  dead  areas  in  which  negligible  acoustic  diffraction  fields  are  present.  The  re¬ 
sults  were  very  attractive. 


Next,  we  note  two  other  new  TSI  architectures.  The  first  system  uses  an  AO  cell  imaged 
onto  a  spatial  mask  with  the  FT  of  the  resultant  signal  produced  being  multiplied  by  a  sec¬ 
ond  spatial  mask  on  which  the  MSF  of  another  signal  was  recorded.  We  have  shown  14]  that 
the  output  from  such  a  system  is  the  correlation  of  a  large  code  equal  to  the  product  of  the 
codes  recorded  on  the  two  Bpatial  masks.  We  have  also  modified  this  system  to  utilize  a  2-D 
mask  and  a  different  vertical  output  detector  system  to  achieve  similar  results  with  more 
flexibility  in  the  code  that  can  be  used  [4].  We  have  analyzed  the  noncoherent  and  coherent 
correlations  obtained  from  such  TSI  architectures  and  verified  that  the  desired  envelope  de¬ 
tection  is  obtained  with  the  full  PG  possible  1 4 ] . 

As  our  fourth  new  AO  application,  we  consider  the  TPP  system  for  processing  a  repeated 
product  code  to  obtain  the  full  PG  of  the  code  and  a  large  range  delay  search.  In  Figure  7, 
we  show  how  this  can  be  achieved.  The  received  signal  s(t)  and  the  two  reference  product 
code  signals  u(t)  and  v(t)  are  fed  to  the  elements  of  the  TPP  as  shown.  We  have  shown  [5] 
that  the  output  plane  pattern  for  such  a  system  contains  axes  that  correspond  to  coarse  and 
fine  time  delays  and  that  the  location  of  the  correlation  peak  thus  indicates  the  target's 
range  to  the  full  range  resolution  possible  with  the  code.  In  addition,  the  PG  of  the  sys¬ 
tem  and  hence  the  SNR  of  the  output  correlation  peak  corresponds  to  the  TBWP  of  the  full 
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product  code.  This  car.  approach  106  in  some  cases. 

6.  ACOUSTO-OPT IC  MATRIX-VECTOR  PROCESSORS 

In  this  section,  we  address  a  new  class  of  AO  processors  that  differ  considerably  from 
the  dedicated  architectures  described  in  Sections  2-5.  This  new  class  of  AO  processor  is 
very  general-purpose  in  that  all  of  the  systems  to  be  described  perform  matrix-vector  opera¬ 
tions.  First,  we  introduce  the  basic  matrix-vector  multiplication  concept  together  with  the 
use  of  an  iterative  feedback  optical  processor  (17).  Next,  we  consider  the  realization  of  a 
deconvolver  with  such  an  architecture  using  the  AO  systems  of  Figures  1  and  2  as  the  basic 
system  elements  [6).  This  corresponds  to  a  matrix-vector  problem  in  which  the  matrix  is 
Toeplitz.  As  our  second  AO  matrix-vector  architecture,  we  consider  a  systolic  array  optical 
processor  using  AO  devices  and  its  use  in  the  multipl ication  of  banded  matrices  and  vectors 
of  large  dimension.  We  also  consider  its  use  in  the  solution  of  large  matrix-vector  prob¬ 
lems  when  the  matrix  is  banded.  Finally,  we  conclude  with  a  new  iterative  matrix-vector 
architecture  using  a  frequency-multiplexed  AO  systolic  array  architecture  and  discuss  its 
use  for  the  solution  of  more  complicated  matrix  problems. 


FIGURE  8  General  optical  matrix-vector  feedback  system 
(using  a  2-D  mask)  (17). 


In  Figure  8,  we  show  a  general  matrix-vector  multiplication  system  (using  a  2-D  mask)  as 
well  as  the  iterative  feedback  architecture  version  of  the  same  system.  We  restrict  our 
attention  to  the  PA  -  PB  -  Pq  portion  of  the  system  for  the  moment.  We  imaae  the 
outputs  from  the  linear  LED  or  LD  input  array  vertically  and  expand  the  output  from  each 
horizontally  to  uniformly  illuminate  the  rows  of  a  2-D  mask  at  Pb.  We  then  integrate  the 
light  leaving  the  separate  columns  of  the  Pb  mask  onto  separate  output  detectors  at  Pc.  if 
we  describe  the  output  from  the  LEDs  as  a  vector  a  and  the  transmittance  of  the  mask  as  a 
matrix  (I  -  B) ,  where  I  is  the  identity  matrix,  tFe  detector  outputs  at  Pc  are  seen  to  be 
the  matrix-vector  product  II  -  B]a  and  a  parallel  optical  matrix-vector  processor  results. 
If  we  then  add  an  external  vector  c  to  this  matrix-vector  product  (and  denote  the  vector 
output  at  iteration  k  by  a(k))  and  then  feed  this  vector  sum  back  to  the  LED  inputs  as  the 
input  a(k+l)  at  iteration  k  +  1,  the  system  is  described  by 
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(I  -  B)a(k)  +  c  »  a(k)  -  Balk)  +  c  =  a(k  +  1).  (8) 

In  steady  state,  when  a(k)  =  a(k  +1)  =  a,  equation  (8)  reduces  to  B  a  =  c  and  the  output 
vector  a  is  the  solution 

a  =  B-1c  (9) 

of  the  matr ix-vector  equation  Ba  =  c.  Many  very  attractive  applications  forsuch  an  itera¬ 
tive  optical  matrix-vector  processor  have  been  described  and  demonstrated  by  us. 

For  our  present  purposes,  we  consider  the  realization  of  similar  operations  using  AO  de¬ 
vices  that  are  more  reliable  and  commercially  available  than  2-D  spatial  light  modulators. 
We  begin  by  considering  realization  of  the  matr ix-vector  product  Ba  =  d,  when  B  is  tri¬ 
diagonal,  i.e. 


b21  b22 
b31  b32  b33 


b42  b43  b44 

b53  b54  b55 


Consider  the  system  of  Figure  9  used  to  realize  the  matrix-vector  product  in  (10) .  In  the 
system  of  Figure  9,  the  output  from  separate  LEDs  are  imaged  through  separate  regions  of  a 
AO  cell  and  onto  separate  detector  elements  in  the  output  plane.  In  16),  we  showed  that  if 
the  time-history  of  the  three  diagonals  of  B  are  fed  to  three  input  LEDs  and  the  time-histo¬ 
ry  of  the  vector  a  is  fed  to  the  AO  cell  ancf  if  the  detector  outputs  are  properly  shifted 
and  summed,  then  the  time-sequential  detector  output  is  the  desired  matrix-vector  product 
Ba  =  d  in  (10).  Addition  or  an  external  vector  c  and  feedback  of  the  detector  outputs  back 
to  the  AO  cell  inputs  enables  this  system  to  solve  matrix-vector  equations  such  as  B  a  =  c 
for  a  =  B'lc.  This  is  indicated  [6]  in  Figure  9  together  with  the  use  of  a  slightly  differ¬ 
ent  and  more  stable  iterative  algorithm  than  the  one  described  in  (8) .  This  systolic  array 
architecture  of  Figure  9  is  a  very  practical  and  general-purpose  AO  matr ix-vector  processor. 

Many  AO  processor  architectures  can  be  used  as  the  basic  element  in  the  system  of  Fiaure 
9.  If  the  SI  or  TI  convolver  of  Figures  1  or  2  were  used,  then  the  output  vector  would  be 
the  deconvolution  of  the  received  signal.  This  can  be  seen  by  writing  the  convolution  as  a 
matrix-vector  multiplication  where  the  matrix  is  Toeplitz  as  was  done  in  |6).  Similarly, 
various  difference  equations  can  be  solved  on  the  system  of  Figure  9  by  first  writing  the 
derivatives  in  the  equation  as  finite  differences  as  we  described  in  16). 

Next,  we  consider  a  new  extension  of  this  AO  systolic  matrix-vector  processor  concept. 
First,  we  note  that  the  bandwidth  and  TBWP  of  the  AO  cell  are  not  fully  used  in  the  system 
of  Figure  9  and  that  the  detector  shift  and  summation  operations  required  are  not  attractive. 
In  our  new  optical  systolic  matrix-vector  AO  processor  of  Figure  10  17) ,  we  use  a  linear 
array  of  LEDs  or  LDs  at  the  input  and  again  image  them  through  selected  regions  of  a  AO  cell. 
However,  now  we  form  the  FT  of  the  product  information  leaving  the  AO  cell  and  we  detect  the 
outputs  on  a  linear  output  detector  array  with  parallel  readout.  Into  the  LEDs,  we  feed  a 
matrix  properly  arranged  with  its  columns  and  rows  multiplexed  as  functions  of  space  and 
time.  Into  the  AO  cell,  we  feed  another  matrix  with  its  columns  and  rows  multiplexed  as 
functions  of  time  and  frequency.  As  we  have  shown  [7),  the  time-sequential  output  as  a  func¬ 
tion  of  time  and  space  from  the  output  detectors  can  be  obtained  in  parallel  and  that  it  cor¬ 
responds  to  the  matrix-matrix  product.  Note  that  in  this  system,  no  output  detector  shift 
and  summation  networks  are  required  (as  was  the  case  in  Figure  9) .  We  also  note  that  the 
time-history  outputs  from  the  detectors  are  available  for  direct  feedback  immediately  into 
the  LED  inputs  or  the  AO  cell  input  for  iterative  operation^.  We  have  extended  this  basic 
concept  to  the  solution  of  matrix-matrix  equations,  the  multiplication  of  three  matrices, 
matrix  inversion,  etc.  17)  followina  the  basic  techniques  used  in  our  prior  optical  systolic 
array  matrix-vector  publications.  We  have  also  addressed  the  pipelining  of  data  and  opera- 
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tions  in  such  systems.  From  the  brief  qeneral  remarks  advanced  above,  we  see  that  this  new 
and  most  general-purpose  type  of  optical  matrix-vector  systolic  array  processor  architecture- 
research  has  many  diverse  applications  and  that  it  represents  a  major  new  area  of  applica¬ 
tion  research  for  AO  devices. 


a  =  a(k) 


f  *  COLUMNS 
t  -  *  =  ROWS 

£  =  £  * 

cm  =  £bmnan 


FIGURE  10  New  frequency-multiplexed  acousto-optic  iterative  matrix- 
vector  systolic  array  processor  architecture  [7). 


7.  SUMMARY  AND  CONCLUSION 

In  these  brief  pages,  we  have  reviewed  the  conventional  AO  processors  that  realize  the 
Fourier  transform,  convolution  and  correlation  operations.  We  have  also  described  ten  new 
AO  processor  architectures.  These  have  used  combined  hybrid  time  and  space-integrating  con¬ 
cepts,  2-D  output  patterns,  multi-channel  architectures,  and  general-purpose  matrix-vector 
systolic  array  concepts.  From  this  brief  summary  of  new  and  advanced  AO  architectures,  it 
appears  clear  that  these  devices  have  a  most  attractive  place  in  future  signal  processing 
applications  and  that  they  can  serve  as  very  general-purpose  processors  as  well  as  being 
able  to  easily  be  configured  into  unique  architectures  for  specific  applications. 
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Frequency-multiplexed  and  pipelined  iterative  optical 
systolic  array  processors 


David  Casasent,  James  Jackson,  and  Charles  Neuman 


Optical  matrix  processors  using  acoustooptic  transducers  are  described  with  emphasis  on  new  systolic  array 
architectures  using  frequency  multiplexing  in  addition  to  space  and  time  multiplexing.  A  Kalman  filtering 
application  is  considered  as  our  case  study  from  which  the  operations  required  on  such  a  system  can  be  de¬ 
fined.  This  also  serves  as  a  new  and  powerful  application  for  iterative  optical  processors.  The  importance 
of  pipelining  the  data  How  and  the  ordering  of  the  operations  performed  in  a  specific  application  of  such  a 
system  are  also  noted.  Several  examples  of  how  to  effectively  achieve  this  are  included.  A  new  technique 
for  handling  bipolar  data  on  such  architectures  is  also  described. 


I.  Introduction 

Optical  processors  that  realize  matrix-vector1  and 
matrix-matrix-  multiplications  represent  a  most  at¬ 
tractive  and  rather  general-purpose  optical  processor 
concept.  Iterative  optical  matrix-vector  processors 
constitute  an  even  more  powerful  version  of  these  basic 
systems. :i  However,  the  practical  realization  of  these 
systems  requires  real-time  and  reusable  2-D  spatial  light 
modulators.1  Since  electrically  addressable  2-D  spatial 
light  modulators  of  adequate  performance  are  not 
presently  readily  available,  attention  has  recently 
shifted  to  systems  using  acoustooptic  (AO)  transducers 
that  realize  matrix-vector  multiplications’’  and  iterative 
optical  matrix-vector  operations.15 

These  recent  architectures"’-'5  are  optical  systolic  array 
processors.  However,  these  initial  architectures  have 
not  yet  fully  utilized  the  frequency-bandwidth  and 
time  bandwidth  products  possible  in  the  AO  devices.- 
In  Sec.  II  we  describe  several  basic  frequency-multi¬ 
plexed  optical  systolic  array  architectures  that  more 
fully  utilize  the  signal  capacity  of  the  system  and  that 
require  simpler  output  detection  systems  than  did  the 
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original  architectures  proposed.  In  Sec.  IV  we  include 
advanced  AO  systolic  array  optical  processor  archi¬ 
tectures  with  attention  to  more  complex  operations  and 
with  attention  to  the  throughput  of  the  resultant  sys¬ 
tem.  A  new  basic  optical  systolic  array  processor  ar¬ 
chitecture  using  a  Fourier  transform  output  plane  with 
the  simultaneous  addition  of  multiple  outputs  on  a 
linear  detector  array  with  parallel  readout  is  pre¬ 
sented. 

Since  different  data  processing  problems  require 
different  operations,  we  select  a  specific  case  study  and 
thus  in  Sec.  Ill  introduce  and  describe  a  new  application 
(Kalman  filtering)  for  implementation  on  an  optical 
systolic  array  processor.  This  case  study  (Sec.  Ill)  is 
used  to  delineate  the  basic  operations  required  on  our 
system.  It  is  important  to  properly  pipeline  the  data 
flow  and  operations  in  a  parallel  processor  to  fully  re¬ 
alize  the  parallel  processing  capability  of  such  a  system 
and  to  determine  the  basic  processing  elements  or 
functions  into  which  a  specific  problem  should  he  for¬ 
mulated  for  proper  implementation  on  such  a  system. 
In  Sec.  IV  we  address  this  issue  with  the  operations 
determined  from  our  Kalman  filtering  problem  as  our 
case  study.  The  new  optical  systolic  array  processor 
architectures  presented  realize  matrix  matrix  multi¬ 
plication.  matrix- matrix  matrix  multiplication,  as  well 
as  matrix  inversion.  Previous  iterative  optical  pro¬ 
cessors  required  various  operations  in  t  he  feedback  loop. 
These  are  usually  implemented  digitally  and  can  hence 
appreciably  slow  down  the  cycle  time  of  the  system.  In 
particular,  bipolar  operations  on  these  systems  require 
the  addition  and  subtraction  of  data  from  one  or  sub¬ 
sequent  cycles  and  hence  require  A  D  and  D  A  con 
verters  plus  digital  data  storage  and  addition.  In  X<-c. 
V  we  advance  a  new  technique  to  accommodate  bipolar 
data  on  such  noncoherent  optical  processors. 
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II.  Basic  Frequency-Multiplexed  Optical  Systolic 
Array  Processors 

By  way  of  review  we  show  the  prior’-'5  optical  systolic- 
system  in  Fig.  1.  This  system  achieves  the  multiplica¬ 
tion  of  a  banded  matrix  B  of  bandwidth  three  hv  a 
vector  a;  e.g.,  B  ■  a  =  d,  or 
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Without  loss  of  generality,  we  have  chosen  the  case  in 
which  the  main  and  the  two  lower  diagonals  of  the  ma¬ 
trix  contain  the  only  nonzero  elements  of  the  banded 
matrix.  To  achieve  this  operation,  the  time-sequential 
input  signals  to  the  three  LEDs  (or  laser  diodes)  and  the 
acoustooptic  cell  are 
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where  time  increases  from  right  to  left  and  where  the  bit 
duration  for  each  element  of  B  and  a  is  7V  As  was 
shown  in  Refs.  5  and  6,  if  the  detector  outputs  are 
shifted  down  at  a  clock  rate  I/Th  and  if  successive 
products  are  summed  in  the  detector’s  CCD  shift  reg¬ 
ister  readout  channel,  the  time-history  output  from  the 
detector  system  will  be  the  desired  matrix-vector 
product  d. 

We  have  examined  various  alternate  arrangements 
of  the  matrix  input  data  B  to  the  linear  LED  arrays  to 
avoid  the  need  for  a  shift  register  summation  readout 
system.  Such  an  approach  is  attractive,  since  quite 
drastically  improved  dynamic  range  will  result  if  the 
original  architecture  of  Fig.  1  is  modified  and  if  proper 
format  control  of  the  input  data  is  used. 

The  system  in  Fig.  1  uses  a  1  -D  input  array  of  LEDs 
or  laser  diodes  and  a  1-D  AO  transducer.  The  philos¬ 
ophy  for  realization  of  an  optical  matrix-vector  product 
on  such  a  system  was  to  represent  the  2-D  matrix  B  by 
the  time  and  space  variables  of  the  LED  array,  i.e.,  we 
represent  the  elements  bmn  of  B  by  b(x,t ).  This  time 
and  space  multiplexing  of  the  input  data  is  quite  at¬ 
tractive  since  the  LED  input  data  must  be  provided  in 


parallel  from  some  source.  As  noted  above,  this  system 
uses  the  AO  cell  only  to  shift  the  input  vector  data  a.  It 
does  not  use  the  large  frequency-bandwidth  and 
time-bandwidth  products  possible  in  AO  cells. 

To  alleviate  these  problems  and  to  produce  a  more 
powerful  optical  systolic  processor,  we  frequency  mul¬ 
tiplex  the  input  data.  In  Fig.  2  we  show  a  simple  initial 
system  to  demonstrate  the  architecture  and  concept. 
We  consider  the  multiplication  of  a  vector  with  elements 
bm  by  a  constant  a.  In  Fig.  2  we  show  a  point  AO 
modulator  (an  AO  cell  with  only  one  spatial  resolution 
element  or  equivalently  an  AO  cell  with  the  input  light 
source  imaged  through  only  1  pixel  of  the  cell).  As  the 
input  to  the  AO  cell,  we  place  each  bm  on  a  corre¬ 
sponding  frequency  carrier  /„,  with  intensity  modula¬ 
tion  proportional  to  bm  and  sum  these  M  terms  to  form 
one  simultaneous  input  signal  br„  (/„, )  to  the  cell  at  1  -bit 
time  Th .  Lens  L  \  now  forms  (at  the  output  plane  P- 1) 
the  Fourier  transform  of  the  light  distribution  leaving 
the  point  modulator.  For  simplicity  we  ignore  the 
bandpass  frequency  response  of  the  AO  cell,  and  we 
show  the  system  for  the  case  of  an  AO  cell  bandwidth 
MAf  extending  from  A f  to  MAf,  where  A f  is  the  fre¬ 
quency  resolution  of  the  system  (corresponding  to  the 
used  aperture  of  the  AO  cell,  i.e.,  the  bit  time  Th  =  1/ 
Af).  At  Pn,  we  place  M  detector  elements  in  a  linear 
array  at  spatial  locations  corresponding  to  these  M 
frequency  components.  At  the  instant  that  the  input 
signal  b,„  (/„, )  is  present  in  the  point  modulator,  we  pulse 
on  the  LED  or  laser  diode  with  an  output  light  intensity 
a.  This  light  intensity  multiplies  the  signal  in  the  AO 
cell  and  M  waves  leave  the  cell  (at  angles  proportional 
to  ).  These  M  wave  fronts  are  focused  onto  the  M 
output  detectors  where  we  obtain  the  M  products  abm 
in  parallel. 

Let  us  next  consider  the  use  of  the  system  in  Fig.  2  to 
realize  the  general  matrix-vector  product  Ba  =  c  or 


To  realize  this  matrix-vector  multiplication  in  the 
system  of  Fig.  1,  we  partition  the  matrix  B  into  column 
vectors  b„  and  write  Eq.  (II)  as 

|b,  . . .  b„  . . .  b.v|a  =  c  (41 

To  realize  Ba  =  c  in  the  form  of  Eq.  (4)  in  the  system  of 
Fig.  2,  we  time  multiplex  the  elements  a„  of  a  as  se¬ 
quential  LED  inputs,  and  we  time  multiplex  each  b„ 
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and  frequency  multiplex  the  M  elements  of  each  b„  as 
the  AO  cell  inputs.  Specifically,  b(  is  the  AO  cell  input 
at  time  7’;t  (with  the  elements  of  b|  frequency-multi¬ 
plexed,  bj  is  the  AO  cell  input  at  time  2Th,  etc.).  We 
choose  to  frequency  multiplex  the  components  of  each 
column  vector  b„  in  Eq.  (4)  since  this  arrangement  al¬ 
lows  us  to  multiply  the  first  element  a,  of  a  bv  all  the 
required  elements  of  B  (e.g.,  bi)  at  time  7V  At  time 
2 7 V  we  multiply  a  >  by  all  the  necessary  elements  bj  of 
B,  etc. 

Several  other  combinations  of  input  data  sequences 
and  different  time-  and  frequency-multiplexing  ar¬ 
rangements  are  also  possible.  Frequency  multiplexing 
of  the  input  LED  or  laser  diode  (or  replacing  it  by  a 
second  point  AO  light  modulator)  are  two  of  many  dif¬ 
ferent  possibilities.  We  selected  the  choice  noted  above 
for  matrix  vector  multiplication,  since  all  M  compo¬ 
nents  of  <■„,  =  '2_nbmna„  are  formed  at  the  proper  M 
output  detector  element.  Each  bmnan  product  is 
formed  at  a  different  (n7’«)  time  interval,  and  thus  we 
can  sum  the  proper  bmnan  products  by  time  integration 
directly  on  each  of  the  M  output  detectors.  This  ar¬ 
chitecture  does  not  require  shift  ing  the  contents  of  the 
output  detectors  and  summing  the  proper  shifted  con¬ 
tents  of  these  detector  elements  (as  was  needed  in  ear¬ 
lier  systems)/’-1’  In  the  system  of  Fig.  2,  after  NTh  time 
intervals,  the  outputs  from  the  M  detectors  at  plane  P:( 
are  the  proper  elements  em  of  the  matrix-vector  product 
Ba  =  c. 

The  arrangement  in  Fig.  2  utilizes  the  bandwidth  of 
the  AO  cell  but  neither  its  full  potential  time  aperture 
nor  its  time  bandwidth  product.  In  Fig.  2  we  show  a 
preferable  topology  for  many  cases.  In  this  architec¬ 
ture,  the  input  vector  a  is  space-multiplexed  rather  than 
time-multiplexed  and  the  matrix  B  is  frc.,  tency-  and 
time-multiplexed  as  before.  In  this  system,  after  NT/t 
time  intervals,  the  contents  of  the  AO  cell  are  as  shown 
in  Fig.  2.  At  this  time  the  LEDs  are  pulsed  on  with  the 
outputs  shown  in  Fig.  2.  This  /V7’p  time  delay  has  en¬ 
abled  the  AO  cell  to  convert  the  time-  and  frequency- 
multiplexed  b(f,t )  representation  of  the  matrix  B  into 
the  indicated  frequency-  and  space-multiplexed  ar¬ 
rangement  shown.  When  the  LEDs  are  pulsed  on  as 
indicated,  the  full  matrix-vector  product  in  Eq.  (2)  or 
Eq.  (4)  is  generated  in  one  Tfi  time  period  in  parallel  on 
the  linear  output  detector  array. 

These  architectures  (Figs.  2  and  2)  are  attractive  and 
preferable  to  the  original  architecture  of  Fig.  1  because 
of  the  increased  flexibility  they  provide  and  because  of 
the  increased  number  of  computations  they  perform  in 


any  time  period  (this  is  necessary  for  such  systems  to 
become  competitive  with  digital  systolic  array  proces¬ 
sors).  The  system  of  Fig.  2  performs  one  scalar-vector 
product  every  TH  and  automatically  adds  the  proper  N 
such  products  on  the  correct  detectors.  It  requires  NTh 
of  time  to  produce  the  product  o  fan  M  X  N  matrix  by 
an  A/  dimensional  vector.  The  system  of  Fig.  2  per¬ 
forms  M  vector  inner  products  of  the  m  th  row  of  B  and 
the  column  vector  a  in  parallel  and  displays  each  of  the 
M  elements  of  the  matrix-vector  product  Ba  on  M 
spatially  separated  output  detectors  in  one  Th  of  time 
(after  an  initial  NTh  delay  time). 

It  is  also  possible  to  frequency  multiplex  each  of  the 
LED  outputs  in  Fig.  2  (and  thus  input  the  matrix  to  the 
LEDs  rather  than  the  AO  cell)  or  to  combine  frequency 
multiplexing  of  both  the  LEDs  and  the  AO  cell.  In  this 
discussion  we  do  not  consider  all  possible  combinations 
of  space,  time,  and  frequency  multiplexing  of  the  dif¬ 
ferent  elements  of  the  system.  The  present  arrange¬ 
ment  in  Fig.  2  is  the  preferred  architecture  for  realizing 
matrix-vector  products  (in  Sec.  IV  we  show  that  this 
system  can  also  perform  many  other  operations).  We 
note  that  the  roles  of  the  matrix  and  the  vector  are  re¬ 
versed  in  this  system  (compared  with  the  one  in  Fig.  1) 
with  the  matrix  represented  by  frequency  and  time 
multiplexing  being  fed  to  the  AO  cell  rather  than  the 
input  LED  array.  In  Sec.  IV  we  consider  iterative 
matrix  algorithms  as  well  as  the  pipelining  and  data  flow 
of  the  system.  In  such  applications  we  will  find  it 
preferable  to  reverse  the  matrix  coding  used  to  facilitate 
data  flow  and  operational  pipelining.  For  now,  our 
initial  concern  is  to  convey  the  concept  of  frequency 
multiplexing  and  the  new  general  optical  systolic  array 
Fourier  transform  architecture  of  Fig.  3. 

If  all  N  components  of  a  are  available  in  parallel,  the 
system  of  Fig.  3  is  preferable.  If  this  is  not  the  case,  the 
system  of  Fig.  2  is  preferable.  We  note  that,  although 
the  system  in  Fig.  3  produces  the  full  matrix-vector 
product  in  parallel,  a  time  delay  of  JVTh  is  required 
before  this  output  is  obtained  (this  is  necessary  to 
properly  format  the  contents  of  the  AO  cell,  using  it  as 
a  time  delay  element).  In  Sec.  IV  we  discuss  how  proper 
pipelining  of  data  and  operations  can  avoid  this  delay. 
We  do  not  consider  the  system  of  Fig.  2  furt  her  since  its 
potential  is  quite  limited. 

The  locations  of  the  detector  elements  in  these  fre¬ 
quency-multiplexed  architectures  are  easily  calculated. 
For  the  RF  frequency  /„,  associated  with  row  m  of  the 
matrix  A,  the  spat  ial  frequency  of  the  signal  in  the  cell 
is  um  =  (ml Pv ,  where  p,  is  the  velocity  of  sound  in  the  AO 
cell.  The  corresponding  location  of  this  frequency 
component  in  the  output  P;i  plane  is  a  distance  d„, 
nmA//./n  =  fm Xf i  hws  below  the  central  point  of  the 
output  Fourier  transform  plane  (where  A  is  the  wave¬ 
length  ot  light,  n  is  the  index  of  refraction,  and  //.  is  the 
focal  lengt  h  of  the  Fourier  transform  lens). 

To  fully  utilize  the  architecture  of  Fig.  3,  we  consider 
the  realization  of  more  complex  basic  operations  beyond 
the  simple  matrix-vector  multiplication  considered  thus 
far  (e.g.,  matrix-matrix  products,  matrix-matrix- 
matrix  products,  and  matrix  inversions).  We  believe 
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that  the  reduction  and  description  of  a  specific  data 
processing  problem  to  such  more  complex  fundamental 
operations  represent  the  best  utilization  of  such  a  par¬ 
allel  architecture.  To  detail  such  operations,  a  specific 
application  is  necessary.  In  Sec.  Ill  we  describe  a  new 
and  most  powerful  general  matrix  processing  problem 
for  such  systems  (Kalman  filtering).  A  discussion  of 
the  realization  of  several  of  the  matrix  operations  re¬ 
quired  in  Kalman  filtering  using  the  architecture  of  Fig. 
.1  follows  in  Sec.  IV,  where  we  also  address  the  associated 
pipelining  and  flow  of  data  and  operations  in  such  a 
parallel  processor. 

III.  Kalman  Filtering  on  an  Iterative  Optical  Systolic 
Array  Processor 

Our  earlier  optical  matrix  -vector  research  on  optimal 
control8"'  used  an  iterative  optical  processor  with  a  2-D 
spatial  light  modulator  mask.  In  such  applications,  the 
refresh  time  (30  msec,  typically)  of  2-D  light  modulators 
restricted  the  algorithms  we  could  consider  to  those 
which  required  very  few  changes  of  the  2-D  mask.  Our 
original  optimal  control  problem  involved  the  solution 
of  the  algebraic  matrix  Ricatti  equation  to  calculate  the 
feedback  gains  and  optimal  controls  to  be  applied.  Our 
proposed  solution8"'  involved  a  double-nested  iterative 
algorithm  with  an  inner  and  outer  iterative  loop.  On 
each  outer  iterative  loop,  the  mask  (the  2-D  matrix)  had 
to  be  altered.  Our  new  architecture  in  Fig.  3  removes 
the  restriction  of  considering  only  algorithms  that  re¬ 
quire  a  minimum  number  of  changes  of  the  matrix 
mask.  Moreover,  our  prior  optimal  control  applications 
were  restricted  to  the  case  when  we  assumed  that  all 
states  in  our  system  model  were  observable  and  that  the 
noise  statistics  were  known  in  advance.  In  practice,  this 
is  not  the  case.  Rather,  we  must  estimate  many  of  the 
states  in  the  control  system  (and  we  must  also  update 
our  estimates  of  the  noise  statistics  after  each  new  input 
measurement).  A  solution  to  these  problems  is  the 
discrete-time  Kalman  filter.  This  is  an  optimal  filter 
in  the  minimum  mean-square  sense. 

In  Table  I  we  summarize  (without  derivation)  the 
basic  discrete-time  Kalman  filtering  equations.1"  All 
control  parameters  used  are  defined  in  Table  II. 
Equations  (la)  and  (lb)  in  Table  I  model  the  system  as 
a  linear  vector  difference  equation.  In  our  discrete-time 
system  model,  we  assume  that  the  system  noise  vector 
w  and  the  noise  vector  v  associated  with  measurement 
k  are  Gaussian  distributed  and  of  zero  mean,  as  in  Eqs. 
( 1  c)  and  ( Id),  and  that  they  are  uncorrelated  as  defined 
in  Eq.  (le).  The  initial  state  estimate  x()  and  the  initial 
error  covariance  matrix  M„  in  Eq.  (If)  complete  our 
system  model.  The  basic  operations  required  in  Kal¬ 
man  filtering  and  state  estimation  involve  an  iterative 
algorithm  (with  the  iteration  index  k  denoting  different 
time  samples).  The  actual  sampling  times  kTs  are 
equally  spaced.  We  assume  for  now  that  the  noise 
statistics  (Q,R)  are  known  a  priori.  The  svstem  model 
(*,r.H)  is  assumed  to  be  known.  Just  prior  to  time 
k 7\ ,  we  know  the  state  x/,  of  the  system  and  the  error 
covariance  M/,  from  the  time  update  Eqs.  ( 1  j)  and  (lk). 
We  also  know  (or  can  precompute  if  the  noise  statistics 
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are  known  in  advance)  the  matrices  P/,  and  K/,  in  Eqs. 
(lg)  and  (lh).  At  time  kTs.  we  make  a  measurement 
z/,-.  We  then  calculate  the  new  state  estimate  x/,  from 
Z/,  as  in  Eq.  (li).  This  estimate  is  the  expected  value  of 
X/,  given  ail  the  measurements  from  |z/,  |  to  z /,..  We  then 
calculate  the  extrapolated  state  estimate  x<,+ ,  as  in  Eq. 
(Ij).  This  is  based  on  the  system  model  and  our  x/,  es¬ 
timate.  If  Q a  and  R/,  are  known  as  assumed.  P/, .  K;, . 
and  M/,  +  i  can  be  precomputed  in  advance  from  Eqs. 
(lg),  (lh),  and  (lk). 

In  this  initial  example  the  only  required  calculations 
are  the  state  estimate  x/,  in  Eq.  (li)  and  the  extrapolated 
state  estimate  X/.+  ,  in  Eq.  ( 1  j ) .  In  this  case.  Kalman 
filtering  requires  only  matrix-vector  multiplications 
and  vector  additions  and  subtractions.  Thus,  the 
simple  system  of  Fig.  1  or  Fig.  2  suffices.  We  address 
how  vector  additions  and  subtractions  can  be  accom¬ 
modated  in  our  system  in  Sec.  V.  For  now  we  return  to 


lie 


APPLIED  OPTICS  Vol  22.  No  1  1  January  1983 


our  Kalman  filtering  discussion  and  we  consider  the 
more  practical  case  when  our  a  priori  noise  statistics 
(Ql.R^>  are  inaccurate  or  not  known  (together  with 
their  time  history)  in  advance.  In  this  case,  after  the 
measurement  z ,  we  calculate  extrapolated  noise  sta¬ 
tistics  Q>,  and  from  our  prior  estimates.  We  then 
calculate  P/,  and  K*  in  Eqs.  ( lg)  and  ( 1  h)  and  finally  X/, 
in  Eq.  (li).  Next  we  extrapolate  the  state  estimate  and 
the  covariance  estimate  to  the  next  sampling  time  using 
Eqs.  ( lj)  and  ( lk). 

In  this  more  practical  case,  Eqs.  (lgl-dk)  must  he 
calculated  for  each  new  input  measurement.  The  op¬ 
erations  in  Eqs.  (lg),  (lh),  and  (lk)  require  the  new 
operations  of  matrix-matrix-matrix  multiplication  and 
matrix  inversion  in  addition  to  matrix  summations.  We 
note  that  the  covariance  matrices  M/,  and  P/,  are  sym¬ 
metric  and  specific  algorithms  can  he  used  to  simplify 
the  computations.  To  maintain  the  generality  of  our 
processor,  we  will  not  exploit  this  matrix  feature  at 
present.  Instead,  we  use  the  matrix  operations  that 
emerge  from  this  Kalman  filtering  application  to  define 
more  general  and  complex  matrix  operations  required 
in  a  typical  problem.  We  now  consider  how  to  realize 
these  operations  on  our  frequency-multiplexed  systolic 
array  processor  with  attention  to  an  efficient  pipelining 
and  flow  of  data  and  operations. 

IV.  Pipelining  and  Flow  of  Data  and  Operations 

In  this  section  we  consider  how  the  advanced  opera¬ 
tions  of  matrix- matrix  and  matrix-matrix-matrix 
multiplication,  matrix  inversion,  and  matrix  addition 
can  he  realized  on  our  basic  optical  systolic  processor. 
To  concentrate  our  attention  on  the  architecture  that 
achieves  the  best  pipelining  and  flow  of  data  and  oper¬ 
ations,  we  suppress  the  details  of  the  required  timing 
and  control  circuits  and  the  required  data  storage  and 
buffering.  In  all  case  studies  to  be  presented,  we  as¬ 
sume  square  matrices  (A/  =  iV)  with  no  loss  of  general¬ 
ity.  To  enable  the  data  and  operational  tlow  to  be  more 
clearly  shown,  we  consider  only  the  case  of  matrices  of 
order  N  =  M  -  3.  Our  system  architectures  can  be 
directly  extended  to  the  case  of  matrices  of  larger 
order. 

We  first  consider  the  basic  space-,  time-,  and  fre¬ 
quency-multiplexed  optical  systolic  matrix-matrix 
multiplication  system  of  Fig.  4  used  to  realize  the  ma¬ 
trix-matrix  product  AB  =  C.  This  architecture  uses 
the  basic  system  of  Fig.  3  with  the  matrix  B  fed  to  the 
linear  input  LED  array  as  bnm  =  b{x,t)\  i.e.,  using  space 
and  time  multiplexing.  The  matrix  A  is  fed  to  the  AO 
cell  as  am„  =  a(/,t),  i.e.,  using  frequency  and  time  mul¬ 
tiplexing  as  before.  The  matrix-matrix  product  in  this 
system  is  thus  realized  as 
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tolic  matrix  matrix  multiplier. 


Table  III.  Time  History  (with  Space  and  Frequency 
Multiplexing)  of  the  Contents  of  the  Components  In  the 
System  in  Fig.  4  for  Matrix-Matrix  Multiplications 
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where  the  space,  time,  and  frequency  multiplexing  are 
shown  explicitly.  To  achieve  the  proper  output,  we 
recall  that  the  AO  cell  converts  time  to  space,  that  its 
transmittance  shifts  in  space  as  time  increases,  and  that 
the  Fourier  transform  lens  converts  frequency  to  space 
in  the  output  transform  plane. 

To  simplify  description  of  the  system  in  Fig.  4,  we 
show  in  Table  III  the  time  history  of  the  space-multi¬ 
plexed  inputs  to  the  LEDs  and  the  frequency-multi¬ 
plexed  inputs  to  the  AO  cell  as  well  as  the  space  and 
time  outputs  from  the  detector  array.  Note  that  time 
proceeds  from  left  to  right  in  this  table  and  that  the 
basic  space  and  time  arrangements  of  the  matrix  input 
B  to  the  LEDs  are  as  before.  Referring  to  Table  III  we 
see  that,  at  time  T ;i,  all  elements  of  t  he  first  column  of 
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A  multiply  b\\,  all  elements  of  the  second  column  of  A 
multiply  b, |,  and  all  elements  of  the  third  column  of  A 
multiply  b  ,(l.  The  proper  terms  in  these  three  vector 
inner  products  are  formed  and  summed  on  the  correct 
detectors  by  the  indicated  choice  of  space,  time,  and 
frequency  multiplexing.  As  Itefore,  there  is  a  time  delay 
of  2 Th  before  we  can  commence  accessing  the  elements 
of  C.  The  philosophy  used  in  this  space,  time,  and 
frequency  multiplexing  is  the  same  as  Indore.  We  form 
the  products  of  the  three  rows  of  A  and  the  first  column 
of  B  to  give  the  first  column  of  C.  We  then  note  that 
the  three  rows  of  A  are  needed  for  subsequent  multi¬ 
plications  by  the  remaining  second  and  third  columns 
of  B.  We  thus  input  these  later  columns  of  B  into  the 
input  LEI)  array  at  successive  times  with  their  spatial 
locations  shifted  bv  1-bit  location  (to  account  for  the 
propagation  of  the  A  data  in  the  AO  cell). 

Thus,  at  time  7'4  we  use  the  contents  of  the  AO  cell 
(the  three  rows  of  A  with  frequency-multiplexed  col¬ 
umns)  to  multiply  the  second  column  of  B  yielding 
outputs  that  are  the  second  column  of  C  at  time  7V  At 
time  TV,  we  similarly  produce  the  third  column  of  C  as 
our  output.  Thus,  in  57V  this  system  performs  nine 
vector  inner  products  (three  at  a  time  in  parallel)  after 
a  2 Th  initial  delay.  We  gate  the  output  detector  ele¬ 
ments  off  during  this  2 T/,  initial  time  delay.  As  shown 
in  Table  111,  the  output  from  this  system  is  the  desired 
matrix-matrix  product  AB  =  C  with  elements: 

.V 

i„  =  V  «>■ 

*  =  i 

In  general,  the  multiplication  of  two  matrices  of  di¬ 
mensions  (Af  X  N )  and  {N  X  M)  in  this  system  requires 
Af  frequencies,  Af  detectors,  (2 N  -  1 )  LEDs,  and  (2 N 
-  1  )T/i  of  time  [of  which  (.V  -  l)T/j  is  dead  or  delay 
time[. 

From  this  brief  extension  of  our  original  system  in 
Fig.  I)  to  the  realization  of  more  complex  operations  such 
as  matrix-matrix  multiplication  as  in  Fig.  4,  we  have  the 
intuitive  feeling  that,  unless  the  data  flow  and  opera¬ 
tions  are  properly  pipelined,  this  architecture  will  not 
achieve  its  full  potential.  We  now  consider  the  case  of 
forming  a  matrix-matrix-matrix  product  (e.g.,  ABD  = 
E)  on  this  system  as  required  in  the  Kalman  filter  for¬ 
mulation  in  Sec.  Ill.  This  proves  to  be  an  excellent 
example  of  how  proper  pipelining  of  the  data  and  op¬ 
erations  can  achieve  a  reduced  overall  delay  time  and 
efficient  data  flow.  The  basic  matrix-matrix  multiplier 
in  Fig.  4  is  again  used  to  realize  this.  The  operat  ion  and 
data  flow  we  use  involve  first  calculating  AB  =  C  and 
then  feeding  back  C  to  the  AO  cell  with  D  now  applied 
to  the  LED  inputs.  This  system  then  forms  CD  = 
ABD  =  E. 

A  simplified  diagram  of  this  architecture  is  shown  in 
Fig.  5.  This  architecture  includes  the  basic  system  of 
Fig.  4  with  the  addition  of  feedback  through  a  sample 
and  hold  (S/H)  unit  to  a  frequency  multiplexing  and 
switch  box.  This  unit  insures  that  the  inputs  to  the  AO 
cell  are  the  matrix  A  during  the  first  three  time  periods 
and  then  the  matrix  C  during  the  next  three  time  peri¬ 
ods.  In  Table  IV  we  show  the  time-history  flow  (as 
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Table  IV.  Time  History  (with  Space  and  Frequency  Multiplexing)  for  the 
Contents  of  the  Components  in  the  System  in  Fig.  5  for  Matrix -Matrix  - 
Matrix  Multiplications 
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functions  of  frequency  and  spare)  of  the  data  inputs  to 
the  LEDs  and  the  AO  cell  plus  the  detector  outputs.  By 
7’;(  =  3Tn,  the  A  matrix  is  present  in  the  AO  cell  and 
now  the  first  column  of  B  is  entered  into  the  lower  three 
LEDs.  At  this  time,  T  i,  the  detector  outputs  are  valid, 
and  at  times  from  T:t  to  T-,  the  detector  outputs  are  the 
associated  columns  of  C  (as  in  Fig.  4  and  Table  III). 
These  detector  outputs  are  delayed  by  one  clock  period 
Ti j  (by  the  S/H  unit)  and  fed  back  immediately  into  the 
AO  cel)  at  times  from  7’.j  to  7V  At  7V,  the  system  has 
completed  the  first  matrix  multiplication  AB  =  C  and 
the  LEDs  are  now  free  to  accept  the  columns  of  D  as 
inputs  (during  77;  to  7V).  This  is  the  precise  time  at 
which  they  are  needed.  From  77;  to  TH.  the  system 
performs  the  final  CD  =  E  matrix  multiplication  and 
the  elements  e,„„  of  E  =  ABD  appear  at  (he  detector 
outputs  from  77;  to  7V-  We  note  that,  in  this  feedback 
architecture,  no  additional  delay  or  dead  time  is  accu¬ 
mulated  after  the  initial  2Tn  loading  of  the  AO  Cell  and 
that  new  data  can  be  fed  to  the  AO  cell  at  T-  and  the 
LEDs  at  7/ 1  if  necessary  in  specific  problems. 

The  two  matrix-matrix  cycles  performed  on  the 
system  are  summarized: 


120  APPLIED  OPTICS  /  Vol  22.  No.  1  /  1  January  1983 


where  we  again  show  the  space,  time,  and  frequency 
multiplexing  used.  The  first  AB  =  C  matrix  matrix 
multiplication  proceeds  as  in  Eq.  (5).  The  C  output  is 
then  converted  to  frequency  and  time/space  by  the 
frequency-multiplexing  unit  in  Fig.  5  as  shown  in  Eq. 
(7a).  The  second  CD  =  E  matrix  multiplication  in  Eq. 
<7b)  proceeds  exactly  the  same  as  in  Eqs.  (5)  and  (7a). 
For  the  N  -  M  =  8  problem,  the  system  performs  the 
eighteen  vector  inner  products,  three  at  a  time  in  par¬ 
allel,  in  a  total  time  of  87’«-  In  general,  for  (M  X  N) 
matrices,  the  system  requires  M  frequencies  and  de¬ 
tectors,  (2 N  -  1)  LEDs  (equal  to  the  time-bandwidth 
product  of  the  AO  cell),  and  (3N  —  \  )Th  of  processing 
time.  The  total  time  delay  or  dead  time  of  the  processor 
has  not  been  increased  over  the  initial  ( N  -  1  )7’/(  time 
delay.  After  this  initial  delay,  all  operations  of  the 
system  are  properly  pipelined  with  no  additional  de¬ 
lays. 

Rather  than  considering  further  pipelining  beyond 
the  matrix-matrix-matrix  multiplication  operation  in 
Fig.  r>  and  Table  IV,  we  now  consider  the  realization  of 
a  matrix  inversion  on  the  basic  optical  system  in  Fig.  4. 
Our  present  purpose  is  to  determine  the  basic  archi¬ 
tecture  of  a  space-,  time-,  and  frequency-multiplexed 
optical  systolic  processor.  In  computing  the  error  co- 
variance  matrix  update  and  the  Kalman  gain  matrix 
update  in  Eqs.  (lg)  and  (lh),  matrix  inversion  is  re¬ 
quired.  This  operation  is  computationally  intensive 
and  is  thus  a  most  attract  ive  one  to  realize  on  our  par¬ 
allel  optical  system.  To  achieve  this,  we  use  a  modified 
Richardson  algorithm"  that  we  previously  described/’ 
Its  realization  on  the  present  system  is  quite  different, 
however.  let  us  consider  solving  C  =  HB  for  B  =  H-lC 
without  explicitly  computing  the  matrix  inverse  H_l. 
This  is  attractive  because  in  some  cases  H  is  ill-condi¬ 
tioned  and  hence  cannot  be  inverted.  To  achieve  this, 
we  use  the  iterative  algorithm  ’ 


Bt ,  i  =  |l  -  u.-H|B,  +  <*•(’.  (St 

where  a-  is  the  acceleration  parameter  that  is  chosen  to 
speed  convergence  of  the  iterative  algorithm.  When  the 
results  B/.  =  B/,  + 1  on  two  successive  iterations  k  and  k 
+  1  are  equal,  Eq.  (8)  reduces  to  the  solution 

B  =  n  T.  (to 

To  realize  this  algorithm  in  our  system,  we  first  re¬ 
write  Eq.  (8)  as 

B,.,,/a-  =  |l/a>  -  H|B,  +  t\  (1(9 

We  then  recognize  that  H  is  known  and  fixed  and  that 
we  can  easily  compute  I /a  by  a  simple  scaling  of  its  el¬ 
ements  (this  flexibility  in  a  variable  acceleration  pa¬ 
rameter  a  on  each  iteration  is  attractive  and  necessary 
for  increased  generality  of  the  problems  for  which  we 
intend  this  processor  to  be  used).  We  thus  write  | I/a 
-H|  as  a  matrix  A  which  we  assume  to  be  known  and 
fixed  (or  easily  recomputed  for  different  a  if  necessary). 
Our  iterative  algorithm  then  becomes 

Bju,  =a|AB,  +  Cl  (11) 

Our  final  general  purpose  space-,  time-,  and  fre¬ 
quency-multiplexed  optical  systolic  processor  archi¬ 
tecture  is  then  as  shown  in  Fig.  6.  This  system  is  similar 
to  that  of  Fig.  5  with  the  addition  of  an  input  unit  that 
computes  |I/a  -  H|  =  A  and  a  resistive  adder  and  am¬ 
plifier  with  gain  a  at  the  output  of  the  detectors. 

In  this  architecture,  the  optical  system  performs  the 
matrix-matrix  multiplication  AB/,,  the  matrix  C  is 
added  one  row  at  a  time  to  the  rows  of  the  AB/.  output 
(in  a  resistor  adder)  to  form  |AB/,  +  C|.  This  matrix 
summation  is  then  multiplied  by  a  to  realize  the  right- 
hand  side  of  Eq.  ( 1 1 )  or  t  he  next  B/,  +  /  iterative  input  to 
the  AO  cell.  In  Table  V  we  show  the*'  '  histories  (as 
functions  of  space  and  frequency)  of  (he  inputs  to  (he 
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AO  cell  and  the  LKDs,  the  outputs  from  the  detectors, 
and  the  inputs  and  outputs  from  the  adder.  For  nota- 
tional  simplicity  in  Table  V,  we  denote  the  elements  of 
the  initial  k  =  0  matrix  B„  by  bmn.  the  elements  of  B| 
(By,  for  iteration  k  =  1)  by  bnin,  and  the  elements  of  Bj 
(for  k  =  2)  by  b„,„.  As  we  see  from  Table  V,  the  initial 
b,nn  inputs  to  the  AO  cell  originate  from  our  initial  es¬ 
timate  as  described  in  Refs.  3,  8,  and  9  and  thereafter 
all  future  AO  cell  inputs  for  all  successive  iterations 
come  from  the  S/H  output  circuit.  Thus,  in  this  final 


architecture  of  Fig.  6,  flow  and  pipelining  of  operations 
and  data  are  ideal  as  there  is  no  dead  or  delay  time  after 
the  initial  (N  -  1  )Th  AO  cell  delay. 

Several  alternate  architectures  were  considered  lor 
this  system.  These  included  feeding  hack  the  output 
to  the  LKDs  rather  than  the  AO  cell  (this  required 
variable  delays  of  the  detector  outputs  with  the  delay 
time  depending  on  the  order  N  of  the  problem  being 
solved).  In  the  system  shown  in  Fig.  6,  a  fixed  1 T a 
delay  (achieved  with  the  S/H  circuit)  is  adequate  for  all 
problems.  Thus,  the  feedback  architecture  indicated 
was  chosen  as  preferable.  Realization  of  the  addition 
of  the  matrix  C  in  the  optical  system  by  including  ad¬ 
ditional  input  LKDs  and  additional  time  was  also  con¬ 
sidered.  Since  this  resulted  in  a  slower  system  with 
poorer  pipelining  of  data  and  operations  and  a  general 
degradation  of  the  modular  structure  of  the  processor's 
architecture,  we  chose  to  achieve  matrix  addition  by  the 
external  resistor  adder  following  the  detector  as  shown 
in  Fig.  6.  On  inspection  of  Table  V  we  see  that,  to  allow 
feedback  of  By,.+ 1  to  the  AO  cell  rather  than  the  LKDs 
(this  is  desirable  as  noted  above),  we  have  used  A  and 
By,,  as  the  inputs  to  the  LEDs  and  the  AO  cell,  respec¬ 
tively.  This  differs  from  the  prior  systems  (Figs.  4  and 
5)  in  which  AB  was  formed  with  A  fed  to  the  AO  cell. 
As  detailed  in  Table  V,  we  achieve  the  proper  matrix 


Table  V.  Data  and  Operational  Flow  and  Pipelining  ol  the  System  in  Fig.  6  lor  Matrix 
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multiplication  and  feedback  by  space  multiplexing  of 
the  rows  of  A  and  time  multiplexing  of  its  columns  at 
the  LEI)  inputs  and  by  frequency  multiplexing  the  rows 
of  B  and  time  multiplexing  its  columns.  This  differs 
from  the  space,  time,  and  frequency  multiplexing  used 
in  Figs.  4  and  ft.  The  AB  =  D  product  thus  appears  on 
the  output  detectors  one  row  at  a  time.  Further  anal¬ 
ysis  shows  that  this  architecture  of  Fig.  (i  can  realize  all 
the  prior  matrix  operations  (matrix-vector  multipli¬ 
cation,  the  solution  of  linear  matrix-vector  equations, 
matrix-matrix  multiplication,  matrix-matrix  matrix 
multiplication,  vector  addition  and  subtraction,  matrix 
addition  and  subtraction,  matrix  inversion,  and  the 
solution  of  linear  matrix-matrix  equations).  YVe  thus 
choose  this  as  our  basic  architecture  for  the  class  of 
problems  we  have  thus  far  considered.  In  later  publi¬ 
cations  we  will  address  the  fabrication  details  of  such 
a  system,  the  partitioning  of  large  matrix  problems  into 
smaller  ones  and  the  full  details  of  implementation  of 
Kalman  filtering  and  other  linear  and  nonlinear  alge¬ 
braic  matrix  applications  on  such  a  processor.  We  now 
address  the  final  topic  of  this  paper:  operat  ing  on  bi¬ 
polar-valued  matrix  and  vector  data  and  the  imple¬ 
mentation  of  matrix  subtraction. 

V.  Bipolar-Valued  Data  Handling 

Many  techniques  have  been  suggested  for  accom¬ 
modating  matrices  and  vectors  with  bipolar-  and  com¬ 
plex-valued  elements  on  an  optical  processor.  We 
consider  only  bipolar  data  representation  here  (the 


curacy  of  the  resultant  processor.  To  maintain  a  high 
system  throughput,  an  alternate  bipolar-valued  data 
algorithm  is  thus  desirable.  We  now  describe  our  new 
solution  to  this  problem. 

We  consider  first  the  multiplication  of  two  bipolar 
scalars  a  and  b.  We  denote  their  positive  and  negative 
parts  by  (n  +  ,o_)  and  (6  +  ,h“),  respectively.  Each  of 
these  components  is  non-negative.  We  recall  that  ifa  + 
>  0,  then  (i  =0,  and  if  a-  >  0,  then  n  +  =  0  ( i.e. ,  half  of 
the  elements  will  always  lie  zero).  The  required  bipolar 
multiplication  is 

(lb  =  id  *  -  (l  Hh  *  -  b  I 

»<«*/»*+( i  b  >  -  ( f/  b 4  F  a  *  h  >  (12) 

From  this  we  see  that  the  positive  and  negative  parts  of 
the  product  ah  are 

[<ih )  *  =  (o  *  /> 4  +  a  b  I.  ( 1  :<a I 

wi/> )"  =  (n  /» 4  +  a  *  />  ).  « 1  :t!>» 

and  the  bipolar  output  product  is 

ah  ~  (ah  I*  —  .  ( 14 i 

whereas  before  only  one  of  the  two  terms  in  Eq.  ( 14)  is 
nonzero.  We  now  extend  this  technique  to  the  case  of 
matrix-matrix  multiplication  AB  =  C  where  the  matrix 
elements  are  bipolar-valued.  We  arrange  each  element 
a„,„  of  the  matrix  as  a  (2  X  2)  submatrix  and  each  ele¬ 
ment  of  B  as  a  two-element  column  vector.  For  .V  =  M 
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extension  to  complex-valued  data  follows  directly  as 
before'1).  In  all  prior  approaches  to  processing  bipo¬ 
lar-valued  data,  the  matrix  was  placed  on  a  bias  (this 
increased  the  dynamic  range  requirements  of  the  de¬ 
tector  and  the  matrix).  In  our  prior  bipolar-valued  data 
algorithm1  we  operated  the  system  twice,  once  with 
positive-valued  input  data  and  once  with  negative- 
valued  input  data.  In  postdetection  electronics,  we 
then  form  the  difference  of  these  two  matrix-vector 
outputs.  This  achieved  the  necessary  bipolar  matrix- 
vector  multiplication.  However,  this  required  consid¬ 
erable  time  (A  D  and  D  A  conversion  and  data  storage, 
plus  a  digital  addition  in  the  feedback  loop).  In  con¬ 
trast,  our  present  architecture  (Fig.  6)  requires  no  A-I) 
and  D  A  conversion  and  performs  the  necessary  matrix 
addition  in  an  analog  system  (this  is  possible  to  suffi¬ 
cient  accuracy  to  be  compatible  with  the  accuracy  of  the 
rest  of  the  processor).  Our  present  initial  concern  is 
pipelining  of  data  and  operations  and  the  speed  of  the 
system.  In  later  publications  we  will  address  the  ac- 


Each  element  r,„„  of  the  output  product  matrix  C  is 
likewise  represented  as  a  two-element  column  vector  as 
shown.  As  before,  only  one  of  the  two  elements  (<•„,„  )4 . 
((',„„)“  will  be  nonzero,  and  all  input  and  output  ele¬ 
ments  will  be  positive  or  zero.  We  have  detailed  this 
new  algorithm  for  the  case  of  matrix  matrix  multipli¬ 
cation  in  Eq.  (15),  since  this  is  the  basic  operation  per¬ 
formed  on  our  new  systolic  processor  shown  in  Fig.  6. 

This  algorithm  leads  itself  to  direct  pipelining  and 
incorporation  into  all  our  architectures  since  the  matrix 
output  is  fed  back  to  the  AO  cell,  and  this  output  C  is 
in  the  required  form  of  the  B  matrix  as  shown  in  Eq. 
(15).  This  increases  the  size  of  t  he  processor  —  1 ) 
LEDs  are  now  required  rather  than  (2 N  —  1 )  as  before 
and  2 M  frequencies  and  2A/  detectors  are  now  re¬ 
quired).  All  previous  bipolar-valued  data  algorithms 
requir?  a  similar  increase  in  the  number  of  point 
sources,  frequencies,  and  detectors,  however.  Our  new 
algorithm  is  superior  to  our  prior  one  because  it  is  di¬ 
rectly  incorporated  into  the  flow  and  pipelining  of  data 
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and  operations  (without  the  requirement  to  operate  on 
output  data  from  successive  cycles)  and  because  it  re¬ 
duces  the  dynamic  range  requirements  of  the  detector 
and  the  matrices. 

The  arrangement  in  Kq.  (15)  can  also  he  used  to 
perform  a  true  matrix  subtraction  without  the  need  to 
handle  negative  numbers.  To  show  this  we  consider 
forming 

A-B<=I)  <  l<i) 

Using  our  prior  (  )+  and  (  )-  notations,  we  write  Kq.  (1(5) 
as 

I)  =  D*  -  D  =  (A*  -  A  )  -  |iB(V  -  (BO  | 

=  |A*  +  (Bn  |  -  |A +  <Bfl*l|.  (17) 

from  which  we  see  that  the  positive  and  negative  parts 
of  the  bipolar  output  matrix  D  are  obtained  as 

D*  =  A*  +  (BCr,  (18a) 

D"  =  A  +  (BO*.  (IHlx 

Realization  of  Kq.  (16)  in  the  form  of  Kq.  (17)  follows 
directly  from  the  matrix  partitioning  used  in  Kq.  (15). 
This  new  bipolar  and  matrix  subtraction  algorithm  now 
completes  our  list  of  operations  required  for  Kalman 
filtering.  A  quite  similar  list  of  operations  exists  for 
many  other  matrix  problems. 

VI.  Summary  and  Conclusion 

In  this  paper  we  have  advanced  and  described  a  new 
iterative  matrix-matrix  application  (Kalman  filtering) 
for  implementation  on  an  iterative  optical  systolic 
processor.  We  have  analyzed  the  operations  required 
to  realize  the  Kalman  filter  and  found  that  they  include 
matrix  matrix  and  matrix-matrix-matrix  multiplica¬ 
tion,  matrix  inversion,  and  matrix  addition  and  sub¬ 
traction.  We  have  proposed  several  new  space-,  time-, 
and  frequency-multiplexed  iterative  optical  systolic 
array  processor  architectures.  We  addressed  the  flow 
and  pipelining  of  data  and  operations  on  these  archi¬ 
tectures  (with  attention  to  the  operations  required  for 
the  Kalman  filter).  The  final  system  in  Fig.  6  repre¬ 
sents  a  general-purpose  basic  optical  systolic  array 
module  capable  of  all  the  required  operations.  We  have 
also  described  a  new  algorithm  for  bipolar  data  and 


matrix  subtraction.  The  resultant  systolic  architecture 
does  not  require  CCD  shift  register  readout  and  sum¬ 
mation  of  the  output  detector  array,  and  it  more  fully 
utilizes  the  frequency-bandwidth  and  time-  bandwidth 
products  of  the  AO  cell.  Our  new  bipolar  data  algo¬ 
rithm  avoids  the  need  for  A-I)and  1)-A  conversion  and 
digital  subtraction  in  the  feedback  loop.  It  thus  pro¬ 
vides  a  fast  system  with  continuously  pipelined  data  and 
operations. 

The  applications  for  iterative  optical  processors  are 
quite  considerable.  The  applications  we  have  ad¬ 
dressed  to  date  include:  adaptive  phased  array  radar 
and  beam  forming,1-  optimal  control,' K-t*  deconvolution 
and  inverse  filtering, •’  and  Kalman  filtering.  Also 
documented11  is  the  use  of  an  iterative  optical  processor 
for  general  operations  such  as  solution  of  simultaneous 
algebraic  equations  and  difference  or  differential 
equations,  least  mean-square  problems,  computation 
of  eigenvalues  and  eigenvectors,  and  nonlinear  matrix 
problems.  The  use  of  iterative  optical  processors  im¬ 
plemented  with  the  new  frequency-multiplexed  systems 
described  in  this  paper  represents  a  most  powerful  and 
general-purpose  data  processing  architecture. 
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Direct  rather  than  indirect  solutions  to  matrix -vector  equations  on  an  optical  systolic  array  processor  are  considered.  A 
frequency -multiplexed  optical  systolic  array  processor  for  matrix -decomposition  is  described.  The  data  flow  and  ordering  of 
operations  for  LU  decomposition  or  gaussian  elimination  and  Lt*  or  Cholesky  decomposition  on  this  system  are  detailed 
using  an  algorithm  that  utilizes  the  parallel  processing  ability  of  the  optical  systolic  array  processor.  The  time  required  for 
this  optical  algorithm  is  found  to  be  much  less  than  for  the  digital  equivalent.  The  data  flow  in  the  optical  tystem  is  seen  to 
be  most  excellent. 


1 .  Introduction 

Optical  matrix -vector  processors  [ 1 ,2)  are  very 
general-purpose  systems  appropriate  for  many  applica¬ 
tions.  The  new  optical  systolic  array  architectures 
[3-51  using  acousto-optic  (AO)  devices  are  even  more 
attractive  because  both  the  vector  and  matrix  are  easily 
updated  in  real-time.  However,  such  processors  require 
attention  to  the  pipelining  and  flow  of  data  and  opera¬ 
tions  [5],  A  primary  application  for  such  systems  is 
the  solution  of  matrix -vector  equations  of  the  form 
Ax  -  6  (or  similar  matrix -matrix  and  nonlinear  matrix 
equations)  [  1  ] .  Thusfar,  only  indirect  or  iterative  al¬ 
gorithms  have  been  suggested  for  the  solution  of  such 
problems  on  optical  processors.  In  this  paper,  we  ad¬ 
vance  a  direct  solution  using  LU  matrix-decomposition 
(or  gaussian  elimination)  [6}  and  also  propose  a  paral¬ 
lel  method  for  Cholesky  decomposition  [6]. 

In  section  2,  we  discuss  such  solutions  and  formu¬ 
late  a  parallel  algorithm  for  LU  matrix -decomposition 
that  is  very  attractive  for  an  optical  realization.  We 
also  note  that  when  direct  techniques  are  used,  it  is 
preferable  to  realize  the  matrix-decomposition  on  an 
optical  system  and  to  utilize  a  digital  processor  for  the 
solution  of  the  simplified  resultant  matrix -vector 
problem.  In  section  3,  we  describe  one  method  of 
realizing  LU  matrix-decomposition  on  a  new  [5]  fre¬ 
quency-multiplexed  optical  systolic  array  matrix  - 
matrix  processor.  In  our  solution,  we  also  note  that 


considerable  attention  must  be  paid  to  the  pipelining 
and  flow  of  data  and  operations  in  any  systolic  array 
processor.  In  section  4,  we  discuss  a  simple  method  for 
extending  the  process  of  LU  decomposition  to 
Cholesky  decomposition  on  our  optical  processor. 


2.  LU  matrix -decomposition 

A  very  popular  direct  solution  to  Ax  =  b  for  x  is  to 
decompose  A  into  the  product  of  a  lower  L  and  an 
upper  U  triangular  matrix.  The  equation  to  be  solved 
then  becomes  LUx  =  b.  One  can  solve  this  equation  by 
first  solving  L y  =  b  for y  and  then  Ux  -y  for  x.  Alter¬ 
natively,  one  can  compute  L_1  and  L~*A  =  b'  and 
solve  Ux  =  b'.  Since  L  and  U  are  triangular  matrices, 
the  solutions  by  back  substitution  are  easily  achieved 
in  dedicated  digital  hardware.  The  computational 
load  associated  with  the  LU  decomposition  is  much 
larger  than  the  solution  of  the  simplified  triangular 
equation  that  results  [6],  Thus,  the  use  of  an  optical 
systolic  array  processor  for  matrix-decomposition 
appears  to  be  a  new  and  most  attractive  application. 

We  now  consider  an  LU  matrix-decomposition  al¬ 
gorithm  that  is  most  suitable  for  implementation  on  a 
parallel  optical  systolic  array  processor.  For  an  ,V  X  .V 
matrix  A,  we  require  A'  -  1  steps.  In  step  1 .  we  form 
M]A  =  Aj  (where  the  first  element  is  the  only  non-zero 
element  in  the  first  column  of  A|).  In  step  2.  we  form 
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M2A[  *  A2  (where  A2  is  such  that  the  first  element  is 
the  only  non-zero  element  in  the  first  column  and  the 
first  two  elements  are  the  only  non-zero  elements  in 
the  second  column).  We  continue  this  procedure  for 
N  -  1  steps  until  we  obtain  A^.j  =  Ayy_,  which 
is  an  upper  triangular  matrix  U.  Each  matrix  M„  is  an 
elementary  lower  triangular  matrix  of  the  general  form 
of  an  identity  matrix  with  non-zero  elements  below  the 
diagonal  only  in  the  nth  column, 


*  A,  we  will  compute  only  U  and  MA  in  (3).  These  cal¬ 
culations  will  be  performed  optically  and  the  simpli¬ 
fied  problem  in  (3)  can  then  be  easily  solved  digitally 
by  back  substitution.  Such  direct  techniques  are  often 
more  attractive  than  indirect  or  iterative  matrix  inver¬ 
sion  algorithms  when  the  same  matrix  is  used  many 
times  (e.g.  in  the  implicit  solutions  of  partial  differen¬ 
tial  equations  as  described  in  [7)).  They  are  also  attrac¬ 
tive  since  the  number  of  steps  required  (N  -  1 )  is  fixed 
and  known.  Conversely,  the  number  of  iterations  re¬ 
quired  in  indirect  solutions  is  not  easily  estimated  in 
advance.  We  assume  that  A  is  either  strictly  diagonally 
dominant  or  positive  definite  so  that  there  is  no  need 
for  pivoting  (interchanging  rows  to  insure 
>a'n;5)forn  +  l  <*<7V)(6). 

3.  Optical  systolic  array  implementation 


where  the  non-zero  elements  of  column  n  of  m„  satisfy 


for  n  +  1  <  k  <  N.  By  the  symbol  a*"'  11  we  denote 
element  (*,  n)  of  A„_  j  at  step  n  -  1 . 

The  product  M„_1 ...  Mj  =  M  is  also  a  lower  triangu¬ 
lar  matrix.  To  solve  Ax  =  A,  we  thus  form  the  upper- 
triangular  matrix  MA  -  U  and  the  vector  MA  *  b'.  We 
note  that  M~ 1  =  L  is  lower-triangular  and  that  A  =  LU. 
Thus,  A x  =  A  can  be  written  as  M-1  Ux  =  b  or  as 


Ux  *  MA 


(3) 


In  our  proposed  LU  decomposition  of  A  to  solve  Ax 


To  optically  implement  the  LU  decomposition,  we 
consider  the  frequency-multiplexed  optical  matrix- 
matrix  systolic  array  processor  [5]  of  fig.  1.  In  this 
architecture,  M  LEDs  are  imaged  through  M  regions  of 
an  acousto-optic  (AO)  cell  and  the  Fourier  transform 
of  the  light  distribution  leaving  the  cell  is  detected  on 
an  output  linear  detector  array.  If  the  matrix  B  is  fed 
to  the  LEDs  with  the  matrix  elements  bmn  encoded  in 
space  x  and  time  t  as  A(x,  t)  and  if  the  elements  amn 
of  the  matrix  A  are  encoded  in  frequency  f  and  time  i 
a sa(f,  r),  then  the  detected  output  C  is  a  matrix  with 
elements  cm„  =  c(x,  f).  This  matrix  is  the  matrix- 
matrix  product  C  *  AB.  If  bnm  =  A(r,  x)  and  amn  = 


rr 


Fig.  1.  Schematic  diagram  of  a  frequency-multiplexed  optical  mauix -matrix  systolic  array  processor. 
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a(t,  f),  then  cm„  =  c(t,  x )  and  C  =  BA  is  produced.  The 
operation  of  this  system  is  detailed  in  (5).  We  denote 
separate  time  slots  on  this  system  in  units  of  a  bit  time 
7g  as  Tj  =  7g,  Ti  =  27"B,  etc.  For  A'  X  N  matrices,  we 
require  (2 N  -  1)  LEDs.  At  each  7g,  N  LEDs  are  used. 
They  are  fed  with  successive  rows  or  columns  of  B.  The 
N  LEDs  used  are  shifted  up  by  one  at  successive  FB 
times.  For  example,  for  bnm  ~  b(x,  f),  the  bottom  N 
LEDs  are  fed  with  the  first  row  of  B  at  TB .  LEDs  2 
through  N  +  2  are  fed  with  the  second  row  of  B  at 
2 TB,  etc.  This  is  necessary  to  allow  the  input  data  to 
properly  track  the  matrix  information  present  in  the 
AO  cell  as  it  moves  through  the  cell. 

To  implement  our  LU  decomposition  algorithm 
described  in  section  2  on  the  system  of  fig.  1,  three 
operations  are  required  at  each  of  the  N  -  1  steps.  At 
step  n,  we: 

( 1 )  calculate  ( 1  /a%~ '*), 

(2)  calculate  the  terms  mkn  =  (-1 /«*"„"  in 

(1)  and  (2)  for  n  +  1  <k<N, 

(3)  calculate  MnA„_i  =  A„  and  M „bn_i=b„. 

After  N  -  1  such  steps,  we  have  our  desired  MiV_1  A/V_2 
=  MA  -  U  upper  triangular  matrix  and  the  M6  vector 
required  in  (3). 

We  perform  steps  (1)  and  (2)  in  simple  analog  elec¬ 
tronics  (fig.  2)  and  perform  step  (3)  on  the  system  of 
fig.  1.  At  successive  TB  times,  the  circuit  of  fig.  2  pro¬ 
duces  successive  rows  of  Mn .  We  denote  row  m  °fM„ 
at  step  n  by  mjj Since  each  row  has  one  element  that 
is  I  and  only  one  other  non-zero  element,  a  simple 
MOS  switching  gate  array  can  select  which  two  LEDs 
are  on  at  each  7g  and  feed  the  1  and  data  to 
these  two  LEDs.  To  form  M„  An_1,  we  thus  feed  suc¬ 
cessive  rows  of  M„  to  the  LEDs  at  successive  times  7~B. 
We  frequency-multiplex  each  row  of  A„_  j  (we  denote 
the  Jfcth  row  by  ~ 1J)  and  feed  successive  rows  to  the 
AO  cell  at  successive  times  TB.  After  ATB  of  time,  the 
full  A„  matrix  is  present  in  the  lower  NT B  time  slots 
of  the  AO  cell.  The  lower  W  LEDs  are  now  fed  with 
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Fi(.  2.  Analog  circuitry  to  compute  the  mth  row  mnm  of  M„ 
at  step  n. 


Table  1 

Detailed  data  flow  for  the  realization  of  LU  matrix -decomposi¬ 
tion  of  a  3  X  3  matrix  on  the  optical  systolic  array  processor 
of  fig.  1 . 


TIME  (T  •  nT.) 

n  B 

T1  T2  T3  T4  TS  !  7t  7  7  ! 

LED  INPUTS 

(FROM  FIC.2  CIRCUIT) 

i  i  1 1  i  i  : 

-1  —2  -3  J  -2  *3  j 

AO  FREO  f  -f 

CELL 

INPUTS  FREQ  f6 

o  o  o  I  1  1  i 

£1  ^lO  f2  -3  | 

h°  k°  K°ir^  h1  h1  ' 

bl  b2  b3|LJ  b2  b3  ! 

DETECTOR  OUTPUTS 

1  1  1 

*1  -2  ~3 

b*  b‘  b>, 

ij 

v,2  b2 

b2  b3 

STORE 

Col .1  of  Aj 
Elem.1  of  bj 

| 

Co!  .2  of  A,  ! 

~2  1 

a33*  ! 

2  332 
b^  *nd  b^ 

and  the  first  rowoj"*  of  A„  is  formed  on  the  de¬ 
tectors.  At  successive  nTB  times,  successive  rows  of 
A„  are  produced.  We  compute  tAnbn_1  =  b„  in  step 
(3)  in  parallel  with  A„  by  adding  an  additional  ( N  + 1  )th 
frequency  to  the  cell  and  encoding  elements  b^~  **  of 
b„_i  on  this  frequency  at  successive  7g  times 

The  nth  column  of  the  final  U  matrix  has  been  cal¬ 
culated  at  step  n  and  at  step  n  + 1  we  do  not  alter  the 
first  n  columns  and  rows  of  A„  or  the  first  elements 
of  b„ .  Thus,  at  each  step,  we  store  the  appropriate  new 
column  of  A„  and  the  corresponding  new  element  of 
bn  and  we  operate  with  matrices  M„  and  A„_]  reduced 
in  order  by  one  on  each  successive  step.  In  table  1 ,  we 
show  the  pipelining  and  flow  of  data  and  operations 
in  the  system  of  fig.  1  for  the  case  of  a  3  X  3  matrix. 
This  table  shows  the  inputs  to  the  LEDs  and  the  AO 
cell  as  well  as  the  detector  outputs  and  the  data  stored 
at  successive  times  T„  =  nTB .  As  before,  we  denote  row 
m  of  and  A„  by  mjjj*  and  and  the  element  m  of 
b„  by  bfg)  (note  that  Aq  =  A  and  b0  =  b). 

For  an  N  X  N  matrix,  we  require  2N  - 1  LEDs, 

N  + 1  frequencies  and  an  AO  cell  of  length  (2N  -  1  )Fg. 
Processing  the  first  column  of  A  requires  (2N  -  I  )FB 
of  time,  processing  the  second  column  of  Aj  requires 
(N  -  1)FB,  for  the  third  column  (N  -2 )TB,  etc.  Ignor¬ 
ing  the  initial  (A^-l)Tg  set-up  time,  the  total  time  for 
an  optica]  LU  decomposition  is 
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l(JV)  +  (Ar-l)+(JV-2)+...  +  2]rB 

~W*+N-2)l2)Tt  (4) 

For  large  N,  approximately  N^/3  multiplications  are 
required  in  the  conventional  serial  digital  LU  decom¬ 
position  approach.  If  we  assume  that  a  multiplication 
time  and  our  bit  time  TB  are  comparable,  then  the 
digital  system  requires  approximately  a  factor  of  ATB 
longer  time  than  does  the  optical  system.  This  occurs 
because  the  optical  system  performs  N  vector  inner 
products  in  parallel  during  each  7"B  time.  Memory 
access  times,  data  management  and  bookkeeping  can 
increase  the  time  required  digitally  (especially  if  A'  is 
large).  As  shown,  data  flow  in  our  proposed  optical 
realization  of  this  LU  algorithm  is  quite  ideal. 

4.  Cholesky  decomposition  and  its  optical 
implementation 

When  a  matrix  A  is  symmetric  and  positive-definite, 
it  can  be  decomposed  into  the  product  of  a  lower- 
triangular  matrix!  and  its  transpose ZT ,  i.e. 

A  =  XXt.  (5) 

This  is  the  Cholesky  decomposition  [6]  and  £  is  the 
square-root  of  the  matrix  A  [8].  This  decomposition 
is  extremely  popular  and  has  many  applications  in 
science  and  engineering  because,  in  many  physical 
problems,  symmetric  (hermitian)  and  positive  definite 
matrices  arise.  We  now  describe  a  new  and  simple 
parallel  approach  for  Cholesky  decomposition  and 
discuss  its  implementation  on  our  systolic  optical 
processor.  The  three  steps  in  the  algorithm  are  . 

(1)  Perform  the  LU  decomposition  on  the  systolic 
optical  processor  as  described  above  to  determine  U. 
This  requires  ((A72  +  N  -  2)/2]  TB  of  time, 

(2)  From  U,  compute  the  diagonal  matrix 


(3)  On  the  systolic  optical  processor  we  then  form 
the  matrix -matrix  product  PU.  This  requires  (2A'-1)7B 
of  time  [5],  This  is  now  our  desired  upper-triangular 
matrix 

£t  =  PU  ,  (6) 

and  the  Cholesky  decomposition  is  uniquely  deter¬ 
mined. 

Our  optical  implementation  of  Cholesky  decomposi¬ 
tion  requires  only  [(A^2  +  5Af  —  4)/2]  TB  of  time.  For 
large  matrices  of  order  N  X.N,  the  conventional 
Cholesky  decomposition  on  digital  computers  take 
approximately  N*/6  multiplications.  Thus,  the  digital 
computation  requires  a  time  approximately  a  factor 
of  ATb/3  longer  than  does  the  optica)  computation. 
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Recently,  there  has  been  increasing  interest1  1  in  extend¬ 
ing  the  repertoire  of  operations  achievable  through  optical 
data  processing.  A  major  class  of  processors  proposed  1  toward 
this  etui  is  that  of  iterative  optical  processors  tlOl'h  The 
generic  101'  has  two  data  paths:  a  forward  path,  where  an 
input  vector  x  is  multiplied  by  a  matrix  M  to  produce  the 
output  vector  y  and  a  feedback  path  which  uses  y  to  produce 
the  input  to  the  forward  path  for  the  next  iteration.  Imple¬ 
mentations  of  lOl’  include  fiber  based  systems1  and  more 
recently  '  systems  based  on  the  use  of  acoustooptic  light 
modulators  along  with  systolic  processing  concepts.  Because 
the  differences  among  these  implementations  are  not  con¬ 
ceptual,  we  will  treat  these  as  equivalent  in  this  better.  Ap¬ 
plications  of  101*  include  the  optima!  weight  vector  determi¬ 
nation  for  adaptive  radar  arrays, 1  matrix  inversion  for  control 
problems,11  and  eigenvalue/eigenvector  estimation." '  In  this 
Letter  we  suggest  a  way  of  using  the  101*  for  estimating  the 
singular  values  and  the  corresponding  singular  vectors  of  a 
matrix. 

The  singular  value  decomposition  (SYD)  of  a  matrix  ,M  of 
size  m  X  n  (with  m  S  n )  is  given  by 

>1  =  Vl)|t ,n 

where  D  is  an  m  X  n  matrix  containing  zeros  everywhere  ex¬ 
cept  along  the  first  r  diagonal  elements  where  r  is  the  rank  of 
the  matrix  M.  Let  us  arrange  the  nonzero  entries  along  the 

diagonal  of  this  matrix  in  descending  order  as  a,.  a  . . ar. 

The  matrices  11  and  V  are  unitary  and  are  of  size  a  x  n  and 
m  x  m.  respectively,  and  the  superscript  T  denotes  the 
transpose  operation.  SYD  was  originally  conceived  as  an 
accurate  method  of  estimating  the  rank  of  a  singular  matrix. 
Andrews  and  Patterson'1  have  shown  its  use  for  image  com¬ 
pression  applications.  SYI)  can  be  considered  as  the  deter¬ 
ministic  counterpart  of  the  statistical  Karhunen-Loeve  (K-Ll 
transform  in  the  sense  explained  below.  The  K-L  transform 
is  based  on  the  computation  of  a  covariance  matrix  from  an 
ensemble  of  images  and  then  estimating  the  dominant  ei¬ 
genvectors  of  this  matrix.  These  dominant  eigenvectors  can 
then  be  shown  to  represent  the  set  of  images  optimally,  but 
a  single  image  may  not  be  represented  well.  'Phus  the  K-L 
transform  has  the  disadvantages  that  it  needs  an  ensemble  of 
images  and  that  it  may  not  represent  a  particular  image  very 
well.  The  SYI)  works  only  on  a  particular  image  and  is  geared 
toward  representing  that  image  in  the  sense  of  minimum 
square  error.  Kven  though  the  usefulness  of  SYI)  for  image 
processing  was  demonstrated  long  ago,'’1"  it  has  not  gained 
popular  acceptance  because  of  the  heavy  computational 
burden  it  imposes.  If  optical  processing  can  be  used  to  per¬ 
form  SYI).  this  computation  can  be  carried  at  a  rapid  rate. 

Because  of  the  heavy  computational  burden  of  performing 
complete  SYI),  it  is  desirable  to  know  a  priori  if  SYI)  is  useful 
for  data  compression  with  a  particular  image.  This  can  be 
answered  by  estimating  the  singular  values  of  a  matrix  and 
then  comparing  the  magnitudes  of  the  few  dominant  singular 
values.  Sahasrabudhe  and  Yaidya"  suggest  a  relation  be 
tween  these  dominant  singular  values  and  the  observed  cor¬ 
relation  coefficient  for  a  Markov  type  image.  Their  method 


has  the  disadvantages  that  the  relation  at  best  is  qualitative 
only  and  that  this  relation  is  true  only  for  Markov  type  images. 
The  method  suggested  in  this  Letter  is  applicable  to  general 
matrices  and  provides  explicit  estimates  of  the  dominant 
singular  values. 

Because  of  the  unitary  nature  of  matrices  l'  and  Y.  we  can 
get  the  following  two  results  from  Kq.  1 1 1: 

(MM  '  IV  =  iVDI'  'all)'  Y'iV 

=  VDiC'CiD'iY'Yi 

=  Vi  1)1) '  I.  eg i 

iM'Ml/'  =  il'l)  'V'nVm'il 
=  LD'iV'ViDil  'l  i 
=  lll)'l)l.  ctl 

From  the  above  results  we  can  easily  conclude  the  following. 
Then  columns  U|,  Uj, . . . ,  u„  of  matrix  l'  are  the  eigenvectors 
of  matrix  M  1  M  with  the  corresponding  eigenvalues  given  by 
the  diagonal  elements  of  D 1  D  of  size  n  Xu.  Similarly  the 
in  columns  vq,  v», ....  v„,  of  V  are  the  eigenvectors  of  t  MM  1  \ 
with  ihe  corresponding  eigenvalues  being  the  diagonal  ele¬ 
ments  of  DDL  Since  D  is  a  matrix  with  diagonal  nonzero 

elements  ah  n  ~ . <r„  the  diagonal  matrices  DD 1  and  D 1  D 

contain  the  nonzero  elements  <rj.  rrj . ni.  Thus  the  singular 

values  a,  of  the  matrix  M  can  be  optically  obtained  by  using 
the  power  method  to  determine  the  eigenvalues  of  the  ma- 
t  rices  MM  1  or  M 7  M  and  taking  the  square  root  of  these 
values.  \Ve  can  also  obtain  the  dominant  left  singular  vectors 
by  determining  the  dominant  eigenvectors  of  MM  r  and  the 
dominant  right  singular  vectors  U  by  determining  the  domi¬ 
nant  eigenvectors  of  M  rM.  More  details  regarding  the  ap¬ 
plication  of  power  method  for  determining  the  eigenvalues  and 
eigenvectors  can  be  found  elsewhere.7  "  This  approach  has 
the  disadvantage  that  the  mask  representing  MM  r  requires 
a  dynamic  range  that  is  twice  (in  decibels)  of  a  mask  repre¬ 
senting  M.  As  pointed  out  by  Stewart.17  this  situation  can 
easily  lead  to  very  erroneous  results  in  the  presence  of  finite 
accuracy. 

A  possible  approach  to  avoiding  the  dynamic  range  prob¬ 
lems  of  the  mask  is  to  use  two  masks,  one  to  represent  M  and 
another  to  denote  M  r.  These  two  masks  can  be  used  in  two 
different  optical  vector/matrix  multipliers  connected  in  a  loop. 
The  functions  of  these  two  matrix/vector  multipliers  can  be 
better  understood  by  looking  at  the  following  results  easily 
obtained  from  Kq.  ( 1 ): 

MY  =  Vl),  cli 

M'V  =  I'D'  mi 

The  above  exact  equations  suggest  the  following  version  of 
power  method  to  estimate  the  singular  values  and  the  corre¬ 
sponding  singular  vectors. 

Step  1:  Choose  an  initial  vector  u,i. 

Step  2:  Determine  the  matrix  vector  product  Mu„.  Norm 
of  this  vector  is  the  initial  estimate  for  the  first  sin¬ 
gular  value.  Normalized  I  Norm  =  1 )  version  of  this 
vector  is  our  new  estimate  for  v.  namely,  this  is  v, . 
Step  2:  Kind  V;.  from  u i,  according  to \q,  =  Norm(Mu/,  j.and 
U(, .  |  is  found  from  vq,  as  per  um  i  =  Norm|M  1  v. , | . 
where  Norm(-|  denotes  the  normalization  such  that 
the  resulting  vector  is  of  unit  length.  The  norm  of 
the  vectors  before  normalization  is  an  estimate  of  the 
singular  values  of  the  matrix. 

Step  4:  Repeat  step  2  until  convergence  is  obtained  in  the 
estimated  singular  values.  The  convergence  be¬ 
havior  of  such  an  approach  is  discussed  else¬ 
where.1' 

As  can  be  easily  seen  from  the  above  algorithm,  the  optical 
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realization  of  this  would  involve  a  lop  half  ol  t he  loop  whic  h 
optically  multiplies  u,,  by  M  to  product-  v,  and  a  holtom  hall 
which  will  multiply  tilt-  v,  venerated  in  the  lop  half,  bv  mat rix 
M'  to  product-  tilt-  vector  u..  (  i  as  the  input  for  tilt-  next  cycle 
tit  the  top  half.  Such  an  arrangement  increases  the  com¬ 
plexity  of  the  hardware  by  a  factor  of  2.  hut  it  enables  us  to 
obtain  more  accurate  estimates  of  the  singular  value  and  the 
singular  vectors. 

The  author  would  like  to  acknowledge  the  support  of  the 
Air  Force  Office  of  Scientific  Research  under  grant 
AFOSR79-0091. 
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ABSTRACT 


The  design,  error  analysis,  component  accuracy 
required,  computational  capacity,  data  flow  and 
pipelining,  plus  the  algorithm  and  application  all 
seriously  impact  the  use  of  optical  systolic  array 
processors.  This  paper  provides  initial  remarks, 
results,  examples  and  solutions  for  each  of  these 
issues. 


1.  INTRODUCTION 

Optical  matrix-vector  processors  11-4]  repre¬ 
sent  general-purpose  optical  processors.  Optical 
computing  has  long  desired  and  needed  such  systems. 
The  iterative  optical  processor  (5-6]  includes  feed¬ 
back  of  the  output  back  to  the  input  of  such  sys¬ 
tems  through  an  analog  (or  digital)  feedback  cir¬ 
cuit.  This  architecture  thus  relieves  the  problem 
of  what  to  do  with  the  output  data  generated  by 
such  systems  and  moreover  it  provides  many  addi¬ 
tional  operations  of  increased  flexibility  and  com¬ 
plexity.  These  architectures  all  require  the  use 
of  real-time  and  reusable  2-D  spatial  light  modula¬ 
tors  (SIMs)  for  the  siatrix  data.  Such  devices  have 
yet  to  become  widely  available.  Hew  engagement 
mode  optical  matrix-vector  processors  [7]  may  be¬ 
come  very  attractive  if  more  advanced  versions  of 
2-D  SIMs  can  be  produced. 

A  store  attractive  and  realisable  optical  line¬ 
ar  algebra  processor  can  be  achieved  using  bulk 
acousto-optic  (AO)  devices.  Several  versions  of 
such  systems  [B-101  have  been  described.  These 
architectures  are  referred  to  as  optical  systolic 
array  processors.  The  most  attractive,  general  and 
powerful  system  of  this  type  is  a  frequency-multi¬ 
plexed  AO  matrix-matrix  systolic  array  processor 
(101.  This  architecture  is  briefly  reviewed  in 
Section  2.  Remarks  on  its  speed  versus  that  of  the 
system  using  a  2-D  SIM  and  associated  remarks  on 
input  data  retirements  are  noted  in  Section  3. 

This  is  felt  to  be  the  most  attractive  optical  sys¬ 
tolic  array  processor  since  it  fully  utilises  both 
the  bandwidth,  aperture  time  and  space  bandwidth 
product  of  an  AO  cell,  since  the  basic  operation  it 
performs  is  a  matrix-smtrix  rather  than  a  matrix- 
vector  operation,  and  since  it  lends  itself  quite 
nicely  to  pipelining  of  data  and  operations. 

The  accuracy  achievable  in  any  optical  or  ana¬ 
log  processor  will  be  limited.  Thus,  the  effect  of 


component  errors  on  such  a  processor  is  a  major 
issue.  Such  analyses  of  conventional  optical  line¬ 
ar  algebra  processors  are  very  limited  (11,12). 

Oir  own  results  have  addressed  the  use  of  such  pro¬ 
cessors  in  the  solution  of  a  total  system  problem. 

In  this  area,  we  have  found  that  for  adaptive 
phased  array  radar  and  optimal  control  that  the  ac¬ 
curacy  of  the  adaptive  weights  in  the  closed-loop 
poles  calculated  on  the  conventional  optical  sys¬ 
tolic  processors  are  sufficiently  adequate.  How¬ 
ever,  other  applications  require  precision  beyond 
1%.  In  such  cases,  optical  convolution  to  realize 
digital  sultiplication  [13]  can  be  incorporated  in¬ 
to  optical  systolic  processors  as  recently  described 
in  Ref. 14,  residue  arithmetic  can  also  be  used  in 
systolic  processors  as  we  recently  described  in 
Ref. 15  or  similar  techniques  can  be  employed  to  re¬ 
duce  the  dynamic  range  requirements  and  improve  the 
accuracy  of  optical  matrix  processors  as  required. 
Space  does  not  permit  us  to  detail  these  issues 
here. 

Next,  we  consider  the  computational  rate  for 
optical  systolic  processors.  The  computing  capaci¬ 
ty  of  an  optical  linear  algebra  processor  must  sig¬ 
nificantly  exceed  that  of  VHSIC,  digital  systolic 
processors,  GaAa  and  other  technologies.  This  is¬ 
sue  is  not  often  detailed.  In  Section  3,  we  dis¬ 
cuss  the  performance  possible  on  the  frequency- 
multiplexed  optical  systolic  array  processor  and  we 
note  that  the  use  of  matrix-matrix  rather  than 
matrix-vector  optical  systolic  processors  repre¬ 
sent  an  obvious  method  by  which  to  achieve 
increased  computing  capacity.  However,  optical  ma¬ 
trix-matrix  processors  which  output  the  entire  ma¬ 
trix  in  parallel  are  not  necessarily  realistic  fcs- 
cause  of  the  data  handling  and  data  flow  problems 
associated  with  feeding  an  entire  matrix  to  a  sys¬ 
tem  in  parallel  or  extracting  an  entire'  matrix  from 
it  in  parallel.  Thus,  data  flow,  pipelining  of 
operations,  data  management  and  storage  require¬ 
ments,  avoiding  A/D  conversion,  etc.  are  often  more 
vital  aspects  of  optical  systolic  processors.  These 
aspects  of  cosqruting  capacity  are  discussed  in  Sec¬ 
tion  5. 

In  Section  4,  we  discuss  algorithms  appropri-  . 
ate  for  realisation  on  such  processors.  The  list 
is  not  complete,  but  attention  should  be  given  to 
the  fact  that  these  optical  systems  should  work  in 
concert  with  digital  processors  and  not  emulate 
digital  linear  algebra  and  digital  systolic  array 
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algorithms.  In  Saction  S,  we  use  tha  aolution  of  a 
matrix  aquation  (or  matrix  inveraion)  aa  an  example 
of  a  moat  vital  iaaua  for  any  ayatolic  array  pro- 
caaaori  data  flow  and  pipelining  of  operationa. 

In  Saction  6,  we  advance  initial  remarka  on  tha  ap¬ 
plication*  for  which  optical  ayatolic  array  proces- 
aora  ahould  be  directed.  Thai#  include  large  di- 
manaionality  prablama  and  problems  Involving  compu¬ 
tationally  complex  matrix  calculationa.  He  alao 
note  that  apecific  caae  atudiea  are  eaaential  to 
detail  the  dcaign,  pipelining  and  performance  of  a 
given  optical  ayatolic  array  procesaor. 

2 .  fREQUENCY-WULTIPLEXED  AO  SYSTOLIC  ARRAY  PROCESSOR 

The  frequency-multiplexed  AO  ayatolic  array 
processor  is  shown  schematically  in  Figure  1.  It 
uses  2W-1  LEDs,  each  imaged  through  a  different  re¬ 
gion  of  an  AO  cell,  with  the  Fourier  transform  of 
the  data  leaving  the  AO  cell  recorded  on  a  linear 
array  of  N  detectors.  To  uae  the  system  for  a  ma¬ 
trix-matrix  aultiplication,  we  consider  the  calcu¬ 
lation  of  A  B  -  C,  where  all  matrices  are  HxN.  We 
frequency  encode  the  elements  of  each  row  of  B  and 
feed  one  row  of  B  to  the  AO  cell  at  each  bit  time 
Tjf.  After  NTg,  the  full  B  matrix  is  present  in  the 
lower  NTb  time  aperature  of  the  AO  cell.  We  then 
pulse  on  the  bottom  N  LEDs  with  the  first  row  a^  of 
A.  The  detector  outputs  are  then  the  first  row  of 
!  C.  This  is  achieved  since  the  product  of  aj  and 

‘  all  elements  of  the  first  column  bj  of  I  leave  the 

[  AO  cell  at  the  same  frequency  f1  and  hence  add  on 

I  the  same  detector  in  the  output  Fourier  transform 

plane.  Siaiilar  remarks  hold  for  the  other  ale stents 
j  of  the  first  row  of  C. 


FIGURE  li  General  optical  systolic  iterative  opti¬ 
cal  processor  architecture  (matrix  in¬ 
version  case  study  detailed) . 


At  each  Tg.  tha  system  calculates  M  vector 
inner  products  (an  K  element  vectors)  in  parallel. 
Frequency-multiplexing  of  the  columns  of  A  into  the 
AO  call  is  also  possible  (and  necessary)  depending 
upon  the  operations  required  (101 .  The  general 
philosophy  of  this  processor  ia  to  time  and  apace 
multiplex  tha  matrix  data  fed  to  the  LEDs  and  to 
tima  (or  space)  and  frequency-multiplex  the  matrix 
data  fed  to  the  AO  cell.  Such  an  architecture  can 
fully  use  the  bandwidth  and  tima  bandwidth  product 
of  the  AO  cell.  Because  one  row  (or  column)  of  the 
output  matrix  is  produced  in  parallel  at  one  time, 
these  outputs  can  be  directly  fed  back  to  tha  AO 


*  ids  .jimwsvr 


call,  thus  data  flow  and  pipelining  is  nearly  ideal 
in  such  a  processor.  We  detail  the  pipelining  of 
data  and  operations  on  this  system  in  Section  5. 

The  basic  operation  in  this  processor  is  a  matrix- 

matrix  multiplication  rather  than  a  matrix-vector  f 

multiplication.  Thus,  this  architecture  performs  N 

times  more  computations  per  bit  time  Tg  than  do  ! 

other  AO  systolic  processors. 

3.  COMPUTING  CAPACITY  1 

The  use  of  an  AO  cell  rather  than  a  2-D  SLM 
does  not  cause  any  loss  of  computing  power  for 
general  matrix  problems.  In  fact,  such  systolic 
processors  provide  Increased  computing  capacity. 

However,  frequency -multiplexing  must  be  employed  to  1 

achieve  this.  Consider  a  Te02  AO  cell  with  a  typi¬ 
cal  aperture  time  of  40ysec  and  a  time  bandwidth 
product  of  10^.  With  the  use  of  frequency-multi-  1 

plexing,  we  can  enter  an  entire  30x3®  matrix  in¬ 
to  the  cell  in  20psec.  Then,  every  TB  »  20/^Q  « 

(iBBrisec  the  system  performs  Sc  vector  inner  prod-  j 

uctt  on  9c  point  vectors  in  parallel  and  displays  ( 

the  outputs  on  the  linear  detector  array.  This 
corresponds  to  100*  operations  per  or  6>x  10^ 

multiplications  and  additions  per  second,  for  a 
typical  multiplication  time  of  0 .  f  M*0&nsec .  Thus 
such  an  architecture  is  very  competitive  with  digi¬ 
tal  processors,  even  if  its  capacity  is  reduced  by  j 

a  factor  of  100f|a 

i 

The  optical  matrix-vector  processors  using  2-D 
SLMs  can  perform  a  comparable  number  of  computa¬ 
tions.  Thus,  no  loss  of  computational  rate  results 
when  AO  systems  are  used.  When  the  matrix  data  in 
the  2-D  SLM  version  of  the  system  must  be  changed, 

30msec  is  typically  required.  ThiB  can  cause  a 
significant  loss  in  effective  computations  and  data 
flow,  since  a  considerable  dead  time  of  30msec  re¬ 
sults  every  time  the  matrix  must  be  changed,  plus 
the  output  data  (and  the  input  data  for  the  matrix) 
must  be  buffered  during  this  time.  Thus,  AO  sys¬ 
tolic  processors  have  more  computing  capacity  and 
flexibility,  than  do  matrix  processors  using  2-D 
SLMs.  In  addition,  the  dynamic  range  of  an  AO  cell 
and  its  uniformity  is  typically  superior  to  such 
parameters  for  2-D  SLMs. 

To  avoid  A/D  conversion  of  the  output  detector 
data  and  to  provide  proper  data  flow,  the  conven¬ 
tional  iterative  algorithms  must  often  be  modified. 

To  demonstrate  this,  we  recall  the  most  popular 
iterative  feedback  algorithsi  (the  Richardson  algo¬ 
rithm)  rewritten  to  solve  the  matrix-matrix  equa-  ; 

tion  HB-  C  for  B  •  2fx£,  i.e.  1 

B(k+1)  -  B(k)  ♦  U[C  -  B  B  (k) ) ,  (1) 

where  k  denotes  the  iteration  index  and  u  is  the 
aceelsration  parameter  used  to  speed  convergence 
(u  is  the  reciprocal  of  the  maximum  eigenvalue  of  H 
which  we  approximate  by  the  Euclidean  norm  of  H) .  ! 

We  rewrite  (1)  as  ~  j 

(B(k+1))/U  -  lI/w-B)B(k)  ♦  C, 

We  realise  that  U/w-H]  is  known  and  easily  calcu¬ 
lated,  since  Z/u  affects  only  the  diagonal  elements 
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of  B  and  the  affact  la  tha  same  for  all  diagonal 
elements.  Danoting  (I/w-H)  by  a  naw  matrix  A,  wa 
rewrite  our  itarativa  algorithm  aa 

B(k-fl)  •  d)(AB(k)  ♦  C) .  (2) 

Tha  ayatam  of  Figure  1  rapraaanta  a  rathar 
ganaral-purpoaa  itarativa  optical  matrix-matrix 
ayatolic  procaaaor.  Ma  now  diacuaa  tha  feedback 
portion  of  thia  ayatam  with  attantion  to  tha  itara¬ 
tiva  matrix  aolution  or  matrix  invaraion  algorithm 
of  (2) .  Tha  LED  inputs  ara  tha  z<*n  of  A.  Thay 
ara  aaaily  ca leu la tad  from  H.  Tha  praprocaaaing 
box  faeding  tha  LEDa  requires  only  one  operational 
amplifier,  one  reaiator  adder  and  an  N  element  MOS 
awitch.  At  tha  detector  output,  one  row  of  AB  ap- 
peara  in  parallel  at  a  time.  Thua,  wa  can  add  tha 
matrix  C  to  thia  matrix-matrix  product  (aa  required 
in  the  right  hand  aide  of  (2)1  one  row  at  a  time. 
Thia  ia  achieved  by  a  sinple  N  element  reaiator 
adder.  Multiplication  by  the  acceleration  param¬ 
eter  ia  likewise  trivial  in  electronica.  Thua,  aa 
ahown,  the  entire  right  hand  aide  of  (2)  can  be 
calculated  on-line  without  tha  need  for  A/D  conver- 
aion,  digital  atorage  or  any  appreciable  feedback 
delay.  Such  combinations  of  analog  electronic  ad¬ 
ders  and  multipliers  properly  combined  with  optical 
matrix-matrix  multiplipliers  represent  vary  excel¬ 
lent  and  powerful  systolic  processors  with  high 
computational  rates  and  excellent  data  flow.  Be¬ 
fore  detailing  the  pipelining  and  data  flow,  we 
consider  parallel  algorithms  and  appropriate  opera¬ 
tions  for  realisation  on  euch  ay stems. 

4.  APPROPRIATE  PARALLEL  ALOORITHMS 

Much  of  the  recant  research  on  optical  sys¬ 
tolic  array  processors  has  considered  the  use  of 
these  processors  to  realise  conventional  linear 
algebra  algorithms  and  various  operations.  Many 
of  these  operations  can  already  be  easily  per¬ 
formed  in  digital  VHS1C  and  digital  systolic  pro¬ 
cessors.  A  typical  example  of  this  issue  arises 
in  the  solution  of  a  linear  aystem  of  equations  by 
direct  (matrix-decomposition)  rather  than  indirect 
(iterative)  methods.  Tha  key  lsaue  we  noted  [161 
is  that  performing  tlie  matrix-decomposition  is  the 
most  computationally  Intensive  operation.  In  Ref. 
16,  we  noted  that  the  solution  of  the  resultant 
lower  triangular  system  of  equations  is  trivial  in 
dedicated  digital  systolic  processors  by  back  or 
forward  substitution  algorithms. 

In  Refs. 16-18,  we  detailed  bow  0*  and  LO  and 
Oioleaky  decomposition  is  possible  or  optical  sys¬ 
tolic  array  processors.  In  cases  when  an  all  op¬ 
tical  solution  is  preferable  (e.g.  for  data  flow 
reaeons) ,  an  optical  solution  of  the  resultant 
lower  (or  upper)  triangular  system  of  equations  is 
possible  as  we  detailed  in  [19} .  The  production 
of  Heseenberg  and  tri-diagcxial  matrices  is  also 
possible  as  we  have  noted  [17] .  Such  solutions 
lead  directly  to  eigenvalue  calculation  applica¬ 
tions  of  these  processors  and  to  the  possible  use 
of  other  optical  systolic  srray  processors  (8-9) 
for  solving  tri-diagonal  matrix  equations.  When 
Toaplits  matrices  result,  an  optical  deconvolver 
(9J  is  appropriate.  The  implicit  and  explicit 


solutions  of  partial  differential  equations  (9,16) 
also  merits  similar  remarks.  Attention  to  parallel 
algorithms  (for  exaaple  the  Householder  algorithm) 
and  algorithms  that  pipeline  quite  nicely  (e.g.  new 
descriptions  of  the  solution  of  lower  or  upper  tri¬ 
angular  systems  of  equations1*)  are  necessary  and  es¬ 
sential  for  the  practical  and  efficient  use  of  this 
new  class  of  processor.  Least  square  solutions  and 
other  iterative  algorithms  that  are  more  appropri¬ 
ate  than  our  general  Richardson  algorithm  (when  the 
matrix  has  special  structure)  exists  as  we  have 
detailed  (20). 

To  detail  the  preferable  procedure  for  the  di¬ 
rect  solution  of  a  matrix-vector  equation,  we  con¬ 
sidered  the  QR  matrix-decomposition  solution  of 
Bx>afor  x.  The  matrix  B  is  generally  decomposed 
into  an  orthogonal  £  and  upper  triangular  R  matrix. 
Me  then  obtain  Since  £T  *£“l,  we  can  solve 

Rx»  ^a  ■  a'  ,  where  £-l«£T  i»  easily  calculated 
from  £.  The  solution  of  Rx  »*’  is  easy  in  digital 
hardware  by  back  substitution,  since  this  corres¬ 
ponds  to  a  lower  triangular  system  of  equations. 
However,  to  best  utilise  such  an  algorithm  on  our 
optical  system,  we  should  never  explicitly  calcu¬ 
late  £  and  £T.  Rather,  we  calculate  R  and  a'  di¬ 
rectly  by  successive  matrix-matrix  and  matrix-vec¬ 
tor  multiplications  of  the  matrix  B  and  the  vector 
a  by  Householder  decomposition  matrices  on  our  sys¬ 
tem  of  Figure  1.  This  yields  s'  directly,  thus 
simplifying  the  entire  problem.  This  also  repre¬ 
sents  a  preferable  use  of  optical  systolic  proces¬ 
sors  in  concert  with  digital  systems  (the  conven¬ 
tional  approach  would  be  to  optically  emulate  cal¬ 
culation  of  R  and  £  from  B) .  Other  new  parallel 
and  revised  linear  algebra  algorithms  are  possible 
for  many  other  applications.  They  are  typical  of 
the  more  appropriate  uses  for  such  optical  processors. 
Me  note  that  such  a  philosoDhy  should  be  pursued  in  all 
future  ootical  systolic  array  processor  algorithms. 
Matrix-decomposition  algorithms  are  a  specific  exasi- 
ple  of  algorithms  which  require  a  new  matrix  mask  at 
each  iteration.  If  realized  on  a  system  with  a  2-D  Silt 
mask,  30msec  would  be  required  for  each  change  of  the 
matrix  or  on  each  iteration.  Thus,  this  type  of  algo¬ 
rithm  is  typical  of  those  that  are  appropriate  for 
realization  on  a  systolic  processor,  rather  than  a  2-D 
SIM  matrix-vector  system.  These  are  also  typical  of 
tha  types  of  algorithsis  this  new  type  of  optical  pro¬ 
cessor  should  address.  Me  note  in  pasaing  that  no  ap¬ 
plication  has  yet  appeared  requiring  the  real-time 
eigenvalue /eigenvector  matrix  calculations  possible 
on  such  systems. 

5.  PIPEL THING  OF  DATA  AMD  OPERATTOMS 

Any  systolic  processor  must  pay  close  attan- 
tion  to  the  flow  of  data  and  operations.  In  Ref. 

10,  this  issue  was  first  noted.  In  Refs. 10, 16-19, 
we  provided  many  examples  of  the  data  flow  and 
pipelining  associated  with  the  realisation  of  vari¬ 
ous  linear  algebra  operations  on  such  processors. 
This  issue  is  vital  for  many  reasons.  First,  once 
data  enters  the  AO  cell,  it  must  be  completely  pro¬ 
cessed,  i.e.  all  calculations  requiring  it  must  be  , 
performed  before  the  data  reaches  the  and  of  the 
cell  (typically  in  40-50ysec).  Furthermore,  as 
soon  as  the  parallel  output  data  appears  at  the 
detector,  something  must  be  done  with  it.  In  our 
algorithms  and  in  our  architectural  studies  of  di¬ 
verse  linear  algebra  algorithms,  we  have  given 
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particular  attention  to  such  issuas.  As  a  most 
approprlata  study  of  this  issue,  we  show  in  Tabls 
1  ths  flow  of  data  into  ths  LEDs  and  tha  AO  call, 
and  the  flow  of  data  from  ths  datsetors  back 
into  tha  AO  call  for  realisation  of  tha  solution  in 
(2)  to  tha  matrix -matrix  equation  H  B  •  C  for  B«ff~lc 
on  tha  systaai  of  Figure  1  usinq  tha  qenaral  feed-  ~ 
back  architecture  shown  in  this  fiqure.  Each 
column  of  Table  1  shows  ths  data  contents  of  tha 
correspondinq  system  elements  at  times  Tj  •  T_,  T2  “ 
*3  •  3Tb  etc.  As  shown,  data  flow  and  pipe¬ 
lining  of  operations  is  essentially  ideal  in  this 
system.  Por  our  detailed  3x3  matrix  example,  we 
see  that  as  soon  as  one  row  of  the  B(ktl)  output 
is  calculated,  it  is  available  for  feedback  direct¬ 
ly  into  the  AO  cell.  He  also  note  that  at  tha 
specific  time  that  this  detector  output  occurs,  tha 
bottom  Tb  of  the  time  aperture  of  the  AO  cell  is 
free  and  available  for  new  input  data. 


TABLE  1:  Data  and  operational  flow  and  pipe¬ 
lining  in  the  syetem  of  Piqure  1 
for  matrix  inversion. 
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6.  APPLICATIONS 

As  our  final  topic,  we  note  the  fact  that  if 
optical  linear  alqabra  processors  are  to  became 
competitive  with  diqitsl  technoloqy,  these  proces¬ 
sors  must  consider  five  key  items; 

1.  The  architectures  for  such  systems  should  pro¬ 
vide  full  processinq  capability,  ns  use  of 
matrix-matrix  sol tipi isrs  rathsr  than  matrix- 
vector  operations  is  a  typical  example. 

3.  The  date  flow  and  pipelining  of  operations  in 
such  architectures  must  be  addressed.  Such 


designs  will  clearly  show  the  superior  data 
management ,  bookkeeping ,  and  avoidance  of  A/D  con¬ 
version  that  is  possible  in  optical  systolic 
processors.  Table  1  is  an  example  of  the  effi¬ 
cient  pipelining  possible  on  such  systems  with 
the  proper  choice  of  the  algorithm  used. 

3.  These  processors  should  be  applied  to  problems 
where  the  accuracy  of  the  final  actual  result 
(not  necessarily  the  matrix-vac tor  or  matrix- 
matrix  product)  is  the  performance  measure 
used.  Our  adaptive  phased  array  radar  and 
closed-loop  optimal  control  exaoples  are  vi¬ 
able  specific  cases  where  such  a  philosophy 

is  most  appropriate. 

4.  The  problems  addressed  on  such  processors 
should  be  of  sufficiently  large  site  (i.e.  the 
order  N  of  the  matrix  should  be  large). 

5.  the  calculations  required  in  the  solution  of 
the  problem  should  be  proportional  to  N^ 

(rather  than  N?  or  M)  ■  He  have  found  matrix-de¬ 
composition  and  Kalman  filtering  to  be  examples 
that  fully  utilise  the  possibilities  of  such 
optical  processors  and  that  represent  appli¬ 
cations  with  high  coaputational  loads  (10,16-19). 

Applications  satisfying  such  constraints  exist  in 
many  araas  and  rspresent  the  class  of  problems  for 
which  such  optical  systolic  array  processors  are 
applicable  and  sxist  appropriate. 
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Nonlinear  local  image  preprocessing  using  coherent  optical 
techniques 


David  Casasent  and  Jiabi  Chen 


Two  coherent  optical  systems  are  described  that  can  realize  local  nonlinear  preprocessing  operators  such  as 
the  Sobel  edge -enhancement  function  in  parallel  on  a  -1)  input  image.  By  local,  we  refer  to  the  size  n|  the 
image  region  rather  than  a  nonstationary  process.  Realization  of  such  operators  using  a  multiple  exposure 
matched  spatial  filter  and  a  computer  generated  hologram  is  discussed.  Kxperimental  results  using  these 
techniques  for  it  x  Hand  a  x  5  Sobel  operators,  u  .-.pet-lively,  are  presented.  Our  techniques  can  he  extended 
to  larger  window  sizes  and  other  edge  enhancement  operators.  These  new  operators  are  achieved  by  novel 
coherent  systems  using  complex  arithmetic  with  magnitude  evaluation  of  the  output  pattern. 


I.  Introduction 

Essentially  all  image  processing  applications  and 
systems  ut  ilize  local  nonlinear  preprocessing  operators 
such  as  median  filters  and  various  types  of  edge-en¬ 
hancement  operators.1  The  computational  load  for 
realizing  these  nonlinear  local  operators  is  quite  high, 
and  although  many  of  them  can  he  realized  to  various 
degrees  of  real  time  in  pipelined  digital  electronics,-’  it 
is  worthwhile  to  consider  the  realization  of  such  local 
operators  on  optical  processors.  This  is  especially  true 
when  such  operators  can  he  combined  with  synthesis  of 
an  optical  matched  spatial  filter  and  when  optical  pro¬ 
cessors  are  being  considered  for  other  portions  of  the 
processing.  Conventional  optical  spatial  filtering  sys¬ 
tems  can  achieve  low-pass  high-pass  bandpass  and 
similar  linear  and  global  spatial  filtering  operations. 
The  results  of  global  operations  differ  from  local  ones 
(e.g.,  the  mean  of  an  entire  image  can  differ  considerably 
from  the  mean  of  each  local  3  X  :)  image  region). 
Nonlinear  preprocessing  operators  provide  improved 
SNH  performance1 over  simpler  linear  edge-en¬ 
hancement  operations  such  as  high-pass  filtering.  Such 
local  nonlinear  preprocessing  operations  are  thus  quite 
different  from  the  conventional  linear  and  global  optical 
spatial  filtering  functions,  '['heir  study  and  realization 
on  an  optical  processor  represent  a  new  class  of  most 
useful  operations  that  we  consider.  We  restrict  atten¬ 
tion  to  nonlinear  edge-enhancement  operators.  Our 
concern  is  not  with  image  distortions  produced  by 
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nonlinear  operators  and  which  edge-enhancement  op¬ 
erator  is  best  (for  a  comparison,  see  Refs.  6, 7,  and  1 ,  pp. 
497-99),  but  rather  we  are  concerned  with  the  design 
and  demonstration  of  several  optical  systems  to  realize 
some  nonlinear  local  operators. 

In  Sec.  II,  we  review  several  possible  nonlinear  local 
operators  and  the  features  of  each.  We  also  provide  a 
brief  summary  and  motivation  for  the  different  kinds 
of  edge-enhancement  operators  that  have  been  found 
useful  for  different  applications.  All  these  involve  some 
form  of  derivative  operation.  Our  discussion  empha¬ 
sizes  the  Sobel  edge-enhancement  operator,  although 
the  techniques  we  propose  can  directly  be  extended  to 
many  nonlinear  local  operators.  We  also  briefly  discuss 
nonlinear  operators  of  larger  window  sizes  (5  X  5)  and 
note  that  many  types  of  operators  are  possible  if  larger 
window  functions  are  available.  In  See.  Ill,  we  describe 
our  new  technique  for  realizing  local  nonlinear  image 
processing  operations  optically.  In  Sec.  IV.  we  detail 
the  optical  realization  of  a  9  X  9  Sobel  operator  using 
multiple-exposure  matched  spatial  filter  techniques  and 
include  experimental  verification  of  our  work.  In  Sec. 
V.  we  consider  use  of  a  computer-generated  hologram 
to  realize  such  operations,  and  we  demonstrate  this 
technique  for  a  f>  X  5  Sobel  operator. 

II.  Local  Operators 

In  the  processing  of  multisensor.  IK.  and  other  types 
of  imagery,  the  edges  in  the  image  are  well  known  to  be 
useful  features  for  target  detection  and  classification. 
Many  types  of  edge  detection  and  edge-enhancement 
operations  have  been  proposed,  hut  little  theoretical 
analysis  has  been  presented  to  address  the  types  of 
edge-enhancement  needed  tor  various  pattern  recog¬ 
nition  applications.  We  briefly  summarize  some  of  the 
available  literature  on  this  topic  we  have  found  to  he 
useful.  This  provides  useful  motivation  for  direction 
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into  the  types  of  edge-enhancement  operations  that  are 
needed.  Any  such  general  treatment  requires  careful 
attention  to  ( It  the  forms  assumed  for  the  spectra  of  the 
signal  and  the  noise  (Is  the  noise  white?  Is  the  same 
spectra  assumed  for  both  the  signal  and  the  noise?):  (?) 
the  pixel  correlation  measure  r ;  for  the  image  (/;  ~  I  for 
a  highly  correlated  image,  i)  «  1  for  an  uncorrelated 
image;  ij  is  closely  related  to  the  correlation  length  and 
correlation  coefficient  of  the  image  data);  and  (8)  the 
performance  measures  used  (e.g..  peak-to-sidelobe  ratio 
of  the  correlation  output  or  SNR  at  the  correlation  peak; 
the  second  measure  is  appropriate  for  image  registration 
applications).  We  consider  the  applications  of  pattern 
recognition  and  image  registration,  and  thus  we  use  the 
performance  measures  of  the  peak-to-sidelobe  ratio 
(PSR)  and  th»  SNR  of  the  output  correlation  plane. 
We  note  that  these  are  significantly  different  from  other 
measures""-7  of  performance  previously  considered  for 
edge-enhancement  techniques  (e.g.,  sharpness  of  the 
edge,  noise  in  the  edge-enhanced  image). 

Let  us  now  recall  some  prior  results  on  the  edge- 
enhancement  preprocessing  required  for  our  applica¬ 
tions.  In  Ref.  8,  it  is  shown  that  if  the  image  spectrum 
is  white  (17  «  1).  maximization  of  PSR  requires  no 
preprocessing.  However,  if  the  image  is  highly  corre¬ 
lated  (i)  as  1),  maximization  of  PSR  requires  the  first 
derivative  of  each  image  to  be  formed  prior  to  correla¬ 
tion.  These  results  were  obtained  assuming  no  noise 
in  the  input  image.  If  noise  is  present,  it  has  been 
shown"  "'  that  maximization  of  SNR  requires  a  deriv¬ 
ative  operation  involving  the  sum  of  the  original  image, 
its  first  derivative,  and  a  mixed  derivative  term.  If  rj 
1  for  the  images  (and  no  noise  is  present),  one  can 
show"  that  maximization  of  PSR  can  be  achieved  by- 
forming  the  second  derivative  of  the  reference  image 
rather  than  the  first  derivative  of  each  image  separately. 
This  follows1-  "'  since  the  correlation  of  the  first  de¬ 
rivatives  of  two  images  with  r]  ss  1  is  the  second  deriv¬ 
ative  of  the  correlation  of  the  original  two  images  and 
since  the  order  of  the  operations  can  be  reversed  (be¬ 
cause  they  are  commutative).  A  more  detailed  inves¬ 
tigation1--"'  shows  that  all  above  results  are  only  ap¬ 
propriate  for  imagery  for  which  the  noise  and  the  signal 
have  the  same  statistics.  We  have  also  shown1--"'  that 
for  highly  correlated  images  (r;  ~  1),  the  necessary 
weighting  to  maximize  SNR  (as  well  as  PSR)  is  weil 
approximated  by  the  |Grad|  operator  (followed  by- 
histogram  reshaping).  Several  of  these  results  are  also 
present  in  Ref.  14  in  a  somewhat  different  form. 

Our  present  purpose  is  not  to  derive  or  discuss  the 
optimum  edge-enhancement  preprocessing  operator 
required  but  rather  to  note  the  nature  of  the  prepro¬ 
cessing  required  in  different  pattern  recognition  and 
image  registration  applications.  From  the  prior  brief 
remarks,  we  have  seen  that  the  problem  being  addressed 
(pattern  recognition  or  image  registration,  target  de¬ 
tection  or  target  location,  etc.)  and  the  nature  of  the 
image  (correlated  or  uncorrelated  pixels,  ~  I  or  1/  « 

1 )  affect  the  optimal  preprocessing  required.  We  have 
also  seen  that  all  of  the  preprocessing  noted  in  Refs. 
8  14  involves  some  form  of  derivative  operator  (first. 


second,  mixed.  Grad,  etc.),  and  thus  attention  to  this 
general  class  of  nonlinear  operators  merits  attention 
with  particular  emphasis  on  the  |Grad|  operator. 

Let  us  next  address  how  such  operations  are  con¬ 
ventionally  performed.  Digital  edge-detection  tech¬ 
niques  generally  involve  the  convolution  of  local  linear 
operators  (li  X  I!  window  sizes  are  common)  with  the 
input  image  /(.v.y ).  These  linear  operators  all  provide 
various  approximations  to  first  or  second  derivatives  of 
the  input  image  with  various  window  masks.  The 
simple  2X2  masks  to  realize  d//d.v  and  <if/<)\  are 
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A  much  better  approximation  to  the  directional  deriv¬ 
atives  is  possible  with  the  8  X  ;i  operators 


■-I 

0 

+  r 

-1  -1  -1 

-1 

0 

+  1 

it 

II  II  II 

_-l 

0 

+ 1_ 

<i\ 

■ 

_+l  +1  +1  _ 

Similar  operators  (e.g..  compass  gradient  masks")  are 
possible  for  realization  of  the  derivatives  in  different 
diagonal  directions.  Convolution  masks  that  approx¬ 
imate  the  Laplacian  are  also  possible.  These  operators 
consist  of  various  approximations  to  the  second  deriv¬ 
ative.  For  example,  the  second  derivative  can  be  esti¬ 
mated  as  the  differences  of  the  first  differences.  For 
image  pixel  ( m.n ).  this  approximation  yields  (for  the 
second  derivative  in  .v) 

AJ/'Av-  =  fUn  -  l./i)  -  L’/tm.a)  +  /On  +  tail.  00 

We  can  realize  Eq.  (8)  by  convolving  the  image  fim.n ) 
with  a  8  X  8  mask  function  such  as 
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One  can  realize  other  types  of  Laplacian  operator  by- 
convolution  with  other  masks  such  as 
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In  Eq.  (5a).  the  8  X  8  operator  shown  is  obtained  by- 
summing  individual  operators  such  as  the  one  in  Eq.  (4). 
In  Eq.  (5b).  the  Laplacian  is  formed  by  differencing 
along  diagonals  rather  than  rows  and  columns.  Many- 
other  tvpes  of  Laplacian  operator  can  be  produced:  e.g.. 
Eqs.  (5a)  and  (5b)  can  be  combined,  or  one  can  average 
the  difference  along  three  columns  or  three  rows.  In 
each  case,  these  8  X  8  operators  are  applied  to  each  pixel 
in  the  input  image,  and  the  central  pixel  in  the  corre 
sponding  8  X  8  region  of  the  input  image  is  replaced  by 
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the  product  of  t  hf  mask  weights  shown  in  Kqs.  ( 1 )  (5) 
and  l  lie  corresponding  pixel  values  in  the  input 
linage. 

These  linear  edge-enhancement  techniques  are 
simple  to  realize  in  a  pipelined  processor.  However, 
they  amplify  hit'll  spatial  frequencies  ( i.e.,  noise)  as  well 
as  the  edges  in  the  image.  Preferable  techniques  em¬ 
ploy  nonlinear  operators  in  which  nonlinear  combina¬ 
tions  of  the  results  of  the  basic  linear  operators  are 
produced.  Figure  1  shows  a  block  diagram1’  for  an 
edge-detection  algorithm  for  the  magnitude  gradient 
operator.  The  output  image  from  the  local  operator  is 
thresholded  to  yield  the  final  edge-enhanced  output 
pattern.  To  describe  these  operators  and  how  they  are 
realized,  a  specific  numbering  procedure  for  the  pixel 
values  a,  in  the  input  image  surrounding  the  general 
input  point  (m.n  )  whose  pixel  value  is  /(m.n )  has  been 
adopted: 
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We  consider  the  |Grad|  operator  (noted  earlier  to  he 
useful  for  preprocessing)  as  realized  by  a  Sobel  local 
operator.1  For  this  operator,  the  output  image  value 
g( m.n)  at  pixel  (ni  ,n )  is 

I  =  I.Y  -  +  V-’l1  < 7 ) 

where  the  local  operators  are 
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Recall  that  the  a,  values  in  Kqs.  (8)  correspond  to  the 
intensities  of  the  pixels  surrounding  the  general  input 
image  point  (m.n)  and  that  the  operation  described  by 
Kqs.  (7)  and  (8)  is  applied  at  each  input  image  pixel. 
Thus  Kqs.  (7)  and  (8)  describe  the  final  image  pixel 
value  for  pixel  (m.n)  as  the  nonlinear  combination  of 
its  surrounding  neighbors.  |Note  that  the  original 
values  of  image  pixel  ( m.n )  is  not  used  in  computing  the 
corresponding  output  pixel  value  and  that  the  sum  of 
the  weights  in  the  Sobel  masks  is  ().|  We  can  thus  de¬ 
scribe  this  realization  of  |Grad|  by  the  two  linear  local 
mask  operators 
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GENERAL  EDGE  DETECTION  APPROACH  (GRAD) 
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Inspection  of  Kq,  (9)  and  comparison  with  Eq.  C2) 
show-  that  the  linear  operators  used  in  the  Sobel  masks 
correspond  to  the  partial  derivatives  in  .v  and  y  averaged 
over  the  image  pixels  in  three  horizontal  and  vertical 
rows  and  columns,  with  the  central  row  and  column 
weighted  most  heavily.  In  Ref.  15  it  was  shown  that  the 
averaging  of  difference  estimates  over  several  rows  and 
columns  of  the  image  pixels  with  a  decreased  weighting 
for  pixels  further  from  the  mask  center  provided  im¬ 
proved  performance.  The  Sobel  operator  described  by 
Kqs.  (7)  and  (8)  includes  both  of  these  desirable  features 
and  is  thus  a  most  attractive  edge-enhancement  oper¬ 
ator.  It  was  thus  chosen  for  our  optical  experiments. 

To  best  describe  the  operation  performed  by  the 
Sobel.  we  write  the  output  function  as 

a  =  1 1 u’lftilr  >,)-'  +  Ill'll  'll)  >-| 1  -.  1 101 

where  (  ),  and  (  K  denote  averages  over  x  and  y.  re¬ 
spectively.  This  formulation  emphasizes  that  each  local 
operator  in  Kqs.  (9)  performs  the  first  difference  ap¬ 
proximation  to  the  partial  derivative  of  /  in  .v  and  y 
averaged  over  y  and  x.  respectively.  A  detailed  deri 
vat  ion  of  Kq.  (10)  follows  directly."’  We  detail  only  the 
Sobel  nonlinear  operator,  but  the  technique  is  directly 
applicable  to  realization  of  many  other  operators  such 
as  the  Roberts,  Prewitt.  Kirsch.  and  others  by  simply- 
changing  the  mask  weights  in  Eq.  (9).  An  alternate 
formulation  is  needed  to  describe  edge-enhancement 
operators  such  as  compass  gradients. 

In  the  optical  realization  of  such  nonlinear  prepro¬ 
cessing  operators,  we  can  easily  use  larger  mask  windows 
(e.g..  5  X  5).  For  such  cases,  the  flexibility  possible 
increases  significantly  as  many  choices  are  possible  for 
the  various  mask  elements.  In  one  quite  simple  5  X  5 
Sobel-type  operator.  Eq.  (9)  would  become 
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Many  other  5  X  5  edge-enhancement  operators  can  be 
formulated.  We  will  refer  to  the  5  X  5  nonlinear  local 
operator  defined  by  Kqs.  (7)  and  111)  as  a  5  X  5  Sobel. 
We  consider  its  optical  realization  in  Sec.  V. 
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As  described  in  Ref.  15.  averaging  over  larger  neigh¬ 
borhoods  (five  rather  than  three  elements)  greatly  im¬ 
proves  the  performance  of  the  operator  when  the  input 
SNR  is  low.  Such  operators  produce  wider  edges,  but 
this  is  not  of  major  concern  in  the  pattern  recognition 
cases  we  consider,  and  edge  thinning  is  also  a  possibility 
if  necessary.  As  noted  in  Ref.  15,  different  weights 
(besides  0,  1,  and  2)  for  the  elements  of  the  masks  can 
also  significantly  improve  performance  of  the  operator. 
Many  other  operations  besides  the  5  X  5  Sobel  in  Eqs. 
(11)  can  thus  be  achieved  by  judicious  choice  of  the 
mask  elements.  One  possible  operator  is  the  realization 
of  two  successive  Sobels  by  one  pair  of  5  X  5  masks. 
Another  possibility  is  the  reduction  of  a  5  X  5  input 
image  region  to  a  5  X  3  region  (in  which  each  pixel  value 
is  the  result  of  a  3  X  3  Sobel  applied  to  the  correspond¬ 
ing  3  X  3  region  of  the  original  input  image)  and  the 
subsequent  reduction  of  these  3  X  3  image  regions  to  one 
pixel  value  that  is  the  Sobel  of  the  3  X  3  set  of  pixel 
values  obtained  from  the  first  Sobel.  This  sequence  of 
two  Sobels  is  one  form  of  the  second  derivative  of  the 
original  image.  Such  operators  should  be  realizable  in 
extended  5X5  masks.  Other  local  masks  can  be  used 
to  achieve  mixed  derivative  operations.  With  the  use 
of  5  X  5  masks,  realization  of  Laplacian  and  similar 
operations  is  also  directly  possible.  In  general,  use  of 
window  operators  of  higher  order  are  more  easily  real¬ 
izable  on  the  optical  systems  we  consider,  and  they 
should  be  able  to  achieve  the  types  of  nonlinear  local 
operators  noted  in  Refs.  8-14  as  desirable. 

III.  Nonlinear  Local  Optical  Operators 

Optical  systems  can  perform  the  linear  spatial  dif¬ 
ferentiation  operation  of  various  techniques.17  The  one 
of  interest  to  us  is  performed  on  an  optical  matched 
spatial  filter  correlator.  The  system’s  output  is  the 
convolution  of  the  input  image /(.r.y)  and  the  reference 
function  h  or  the  impulse  response  of  the  system.  (The 
complex  Fourier  transform  H  of  h  is  recorded  on  the 
matched  spatial  filter.)  When  h  is  two  delta  functions 
separated  by  d,  the  system’s  output  is 

output  =  f  •  h  =  fix .V I  •  [/M  y  +  rf.v)  —  <'X.r.vl|  —  ilf/dx .  ( 121 

or  the  first-difference  linear  two-point  approximation 
to  the  1-D  spatial  differentiation  of  the  input  image 
/(jc,y).  This  is  still  a  linear  operation  and  to  realize  the 
Sobel  or  other  such  nonlinear  operators,  we  must  de¬ 
velop  new  optical  techniques. 

W'e  begin  bv  rewriting  the  Sobel  output  function  in 
Eq.  (7)  as 

Clm.nl  =  t.V-  +  V-)1  -  =  1  |-Y  +  yV|T  -  =  |.Y  +  ./>j  il:ll 

This  complex  arithmetic  description  is  most  useful. 
Should  a  digital  processor  capable  of  direct  complex 
arithmetic  in  one  pass  be  developed  and  implemented, 
one  can  realize  the  Sobel  operator  as  in  Eq.  ( 13).  This 
is  not  available  in  present  digital  systems;  however,  an 
optical  system  to  realize  Eq.  ( 13)  is  possible.  The  light 
amplitude  distribution  at  the  output  of  this  optical 
system  will  be  given  by  Eq.  ( 13).  After  detection,  the 
square  of  Fk|.  ( 13)  will  lie  produced  in  an  optical  system. 
This  represents  no  significant  difference  since  the 
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Fin.  -  Multiple-exposure  Sobel  filter  synthesis  system:  M.  mirror: 
VBS.  variable  beam  splitter;  Cl,,  eolliniation  lens  system;  FT  lens. 

Fourier  transform  lens;  MSF.  matched  spatial  tiller. 

square-root  operation  in  Eq.  (13)  is  often  omitted  in  the 
digital  realization  of  such  operators  with  no  adverse 
effects.1’  This  issue  is  not  essential  for  preprocessing 
or  for  the  realization  of  a  nonlinear  filter.  We  thus 
consider  the  realization  of  an  optical  system  whose 
output  amplitude  at  each  point  is 

A’  +  /)"  =  [trr-j  +  2r/.i  -t-  a  jt  —  tr/.i  +  2  a-  ■+ 

+  ;j«i„  +  2n i  +  u  .t  -  (a,,  +  2(i-,  +  <i 4 1 J .  (Ill 

where  a,  are  the  pixel  values  of  the  input  image  in  Eq. 
(6).  We  describe  Eq.  (14)  as  in  Eq.  (12)  by  the  convo¬ 
lution  of  the  input  function  /(.v.v)  with  a  sum  of  delta 
functions  at  eight  spatial  locations  with  complex-valued 
weights  for  each  delta  function: 

,i.’(.v..v I  =  / t.v.vl  •  t.v  -  md.y  -  lull  tl'iat 

=  fix.}  I  •  |l  I  +  yliit.v  -  t/.\  -  ill  +  gnu  -  d.\  I 

+  (1  -  J  h1(.v  -  </,\  +  </ 1  +  2./«t.v.y  -  </) 

-  2/nt.v +  t/l  -  ( 1  -  finer  +  (/,'  -  c/I 

—  2(M.v  +  </.v)  -  ( 1  +  I  l/ll  y  +  </,v  +  ,/l|.  I  jab! 

where  d  is  the  spacing  between  pixels  in  the  input  image 
and  where  the  weights  and  locations  of  each  delta 
function  are  obtained  from  Eq.  (14). 

Our  proposed  technique  for  the  optical  realization  of 
nonlinear  local  preprocessing  operators  such  as  the 
Sobel  is  thus  to  employ  an  optical  correlator  with  a 
matched  spatial  filter  whose  impulse  response  is  a 
weighted  sum  of  delta  functions.  The  necessary 
weights  are  complex-valued.  However,  we  can  achieve 
these  by  holographic  techniques  and  appropriate  phase 
shifting  as  we  detail  in  Sec.  IV.  We  can  also  achieve  the 
required  impulse  response  by  use  of  a  computer-gen¬ 
erated  hologram  as  we  detail  in  Sec.  V. 

IV.  Realization  Using  Multiple-Matched  Spatial 
Filters 

The  desired  impulse  response  in  Eq.  (15)  could  be 
achieved  by  forming  the  holographic  matched  spatial 
filter  (MSF)  of  an  input  function  containing  delta 
functions  (apert  ures)  at  the  correct  locations  and  of  the 
correct  radii  (to  adjust  the  intensity)  and  with  the 
necessary  phase  factors  achieved  by  placing  X/4  or  A/2 
plates  behind  the  appropriate  apertures.  We  chose  to 
form  the  desired  MSF  by  eight  multiple  exposures  on 
one  plate  (because  the  necessary  light  levels  and  expo¬ 
sure  times  were  easier  to  achieve).  The  MSF  synthesis 
system  used  is  shown  in  Fig.  2.  It  is  a  conventional 
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Fig.  A.  System  to  produce  the  Sobel  edge-enhanced  image  from  a 
multiple-exposure  matched  spatial  filter. 

holographic  MSF  system  with  the  addition  of  the  wedge 
and  the  FI'  lens.  The  FT  lens  allows  a  lensless  MSF  to 
be  recorded  (the  MSF  plus  an  FT  lens  with  focal  length 
/;.=/  =  250  mm)  as  we  have  described  and  used  previ¬ 
ously.1*  We  ignore  this  FT  lens  for  our  present  dis¬ 
cussion. 

The  MSF  recorded  at  each  exposure  is  the  interfer¬ 
ence  of  two  plane  waves  and  thus  produces  a  sine  wave 
grating  whose  impulse  response  is  a  delta  function.  The 
amplitude  of  each  delta  function  is  set  by  varying  the 
exposure.  This  does  not  alter  the  modulation  of  the 
grating  being  recorded,  since  the  bias  level  is  also 
changed  for  each  exposure;  however,  the  amount  of  light 
diffracted  into  the  output  correlation  plane  does  vary 
proportionally  to  the  exposure  as  is  desired.  The  lo¬ 
cation  of  each  delta  function  was  controlled  by  trans¬ 
lating  the  FT  lens  in  its  plane  (and  thereby  varying  the 
spatial  frequency  of  the  recorded  grating).  We  pro¬ 
duced  the  necessary  complex-valued  strength  for  each 
delta  function  by  shifting  the  wedge  in  1-D  in  its  plane. 
After  synthesis  of  this  Sobel  MSF  in  Fig.  2,  we  place  it 
in  Fig.  3  and  in  the  output  plane  obtain  the  Sobel  of  the 
input  image. 

We  now  detail  these  systems  for  the  case  of  a  multiple 
MSF  to  realize  the  3  X  3  Sobel  of  Eq.  (14).  All  lenses 
in  Fig.  2  had  //.  =  760  mm.  In  the  Sobel  correlator  (Fig. 
3),  the  first  FT  lens  used  //.  =  500  mm,  and  the  focal 
length  of  the  lensless  MSF  was  /=//,=  250  mm.  This 
correlator  reconstruction  system  exhibited  a  2:1  dem¬ 
agnification.  The  highest  input  spatial  frequency  we 
set  the  system  to  handle  was  um  =  20  cycles/mm.  The 
spatial  frequency  plane  in  Fig.  3  must  thus  have  an 
aperture  of  2 Xf/.Um  =  12  mm.  We  used  15-mm  diam¬ 
eters  for  all  beams  in  Fig.  2  and  for  the  corresponding 
diameter  of  the  MSF.  To  select  the  distance  d  between 
the  delta  function  impulse  responses  (and  similarly  the 
amount  d  by  which  to  translate  the  FT  lens  between 
exposures),  we  determine  that  the  smallest  edges  of 
interest  in  the  8-  X  8-mm-  input  image  were  of  0.2-mm 
width.  In  the  output  plane  of  Fig.  3,  they  would  be  0. 1 
mm.  We  chose  a  spacing  d  =  12.5  pm  between  delta 
functions  and  a  corresponding  12.5-pm  FT  lens  shift. 
This  produced  edges  in  the  output  of  25-pm  width. 
This  will  produce  sufficient  detail  to  provide  a  good 
edge-enhanced  output.  Note  that  in  this  optical  Sobel 
system,  one  can  easily  choose  the  filter  to  select  the 
minimum  input  edge  size  we  wish  to  consider. 

The  eight  exposures  in  Eqs.  (15)  were  arranged  so 
that  successive  exposures  had  a  phase  difference  of  ir/4. 
We  now  compute  the  amount  by  which  the  wedge  in  Fig. 
2  must  be  translated  between  exposures  to  provide  a  jr/4 


or  A/8  phase  shift.  The  wedge  angle  was  4  sec  of  arc, 
and  thus  shifting  it  caused  minimal  displacements  in 
the  input  beam  and  thus  did  not  cause  misalignment  of 
the  system.  To  measure  the  angle  o  of  the  wedge,  we 
measured  the  fringe  spacing  d\  produced  by  interfer¬ 
ence  from  the  front  and  back  surfaces  and  found  dt  = 
(A/2  )Um  =  10  mm,  where  n  is  the  index  of  refraction  of 
the  wedge.  If  the  wedge  is  shifted  vertically  by  a  dis¬ 
tance  du ,  the  beam  traveling  through  it  sees  a  path 
difference  ad„  (n  -  1)  from  its  prior  value.  Substi¬ 
tuting  for  o  and  equating  this  path  difference  to  A/8,  we 
find  that  a  shift  d„  =  7.5  mm  provides  the  desired  phase 
factor. 

The  above  procedure  was  followed,  and  eight  multiple 
exposures  of  the  MSF  plate  were  made.  After  devel¬ 
opment,  we  inspected  the  impulse  response  of  the 
lensless  MSF  and  found  it  to  be  as  required.  We  then 
placed  this  MSF  in  the  lensless  correlator  of  Fig.  3.  A 
converging  beam  system  (with  the  input  behind  the  first 
FT  lens)  was  not  possible  since  such  a  system  was  not 
used  during  synthesis  of  the  MSF.  The  additional 
phase  term  introduced  in  the  converging  beam  system 
would  thus  not  be  canceled  by  this  MSF.  If  the  angle 
of  the  input  plane  wave  in  Fig.  2  were  altered,  the  Sobel 
version  of  the  input  image  would  appear  on-axis  in  Fig. 
3  rather  than  at  an  angle  6  =  15°  to  the  optical  axis  as 
shown.  The  input  image  size  used  was  8X8  mm2,  al¬ 
though  larger  image  sizes  could  be  accommodated  in  the 
system  used.  An  input  text  image  and  the  corre¬ 
sponding  Sobel  edge-enhanced  |  Grad  |  output  image 
obtained  on  this  system  are  shown  in  Fig.  4.  The  results 
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Pig.  1.  Input  irnagr  la)  and  Sobel  edge-enhanced  image  (b)  using 
the  multiple-exposure  matched  spatial  filter  system. 
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appear  as  expected.  All  input  lines  are  >2</  in  width 
and  thus  have  output  image  edges  of  width  'Id. 

V.  Realization  Using  a  Computer-Generated 
Hologram 

It  is  possible  to  produce  the  necessary  FT  plane  filter 
function  for  the  Sohel  (or  other)  edge-enhancement 
operators  by  use  of  computer-generated  holograms 
(CGH).1'  The  general  form  for  the  desired  impulse 
response  for  a  Sohel  operator  is 


1.  1_  ib. v  -  md.\  -  ik/  l 


I  HU 


be  lost  in  the  reconstructed  output  from  this  filter.  A 
different  CGH  synthesis  facility  would  provide  less 
image  resolution  restrictions. 

The  CGH  corresponding  to  Kqs.  ill)  was  computed, 
plotted,  and  photoreduced  as  described  above.  Il  was 
then  placed  in  the  FT  plane  of  the  system  of  Fig.  a  with 
the  local  length  of  the  first  lens  being  ">00  mm  and  that 
of  the  second  lens  being  1 15  mm.  The  input  and  output 
images  obtained  for  an  aerial  input  photograph  are 
shown  in  Figs,  ti(a)  and  (b).  For  comparison,  the  output 
from  the  5  X  5  multiple-exposure  MSF  Sohel  applied 
(o  this  same  image  is  included  in  Fig.  tile).  The  higher 
resolution  of  the  5  X  :l  Sohel  is  apparent  as  are  the 
broader  lines  present  in  the  5  X  5  Sohel  output.  For  a 
5  X  5  Sohel  and  an  input  with  edges  of  Id  width,  we 
expect  the  edges  in  the  output  to  he  :id  wide  (compared 
with  2d  for  the  5  X  5  Sohel).  Our  data  verify  this  since 
all  edges  in  the  output  from  the  5  X  5  Sohel  | Fig.  (Kb)] 
are  wider  than  those  in  Fig.  (lie)  (obtained  using  a  5  X 
;i  Sohel).  The  edges  that  are  not  present  in  Fig.  G< b t 
were  (in  general)  lost  by  the  limited  5-cycle/mm  maxi¬ 
mum  bandpass  of  the  CGH. 


To  realize  such  an  impulse  response  with  a  CGH.  we 
Fourier  transform"1  sample  it.  and  for  each  sampling 
cell  in  the  CGH  we  record  an  amplitude  A  and  a  phase 
</>.  In  ( 16).  we  ignore  (for  simplicity  of  the  expressions 
presented)  the  fact  that  this  CGH  is  an  off-axis  holo¬ 
gram.  The  transmittance  of  the  FT  plane  CGH  is  thus 
of  the  general  form 

.tl.v.\  l  exp|/ oU,\  l).  1 17) 

where  U  .v)  in  ( 17)  refers  to  the  coordinates  of  the  FT 
plane  where  the  CGH  will  be  placed.  'Phis  CGH  filter 
technique  is  quite  appropriate  for  Sobei  operators  with 
large  neighborhoods  (i.e..  5  X  5).  since  25  multiple  ex¬ 
posures  are  quite  difficult  to  achieve  with  the  holo¬ 
graphic  MSF  system  of  Fig.  2.  As  discussed  earlier, 
such  filters  are  much  more  flexible  and  are  of  significant 
importance  when  the  input  has  a  low  SNR. 

We  chose  to  demonstrate  this  technique  for  the  5  X 
5  Sohel  in  Kqs.  (11).  We  now  describe  the  design  and 
fabrication  of  the  CGH.  The  CGH  contains  I.  X  W 
cells,  each  of  size  dx  X  dy  and  each  containing  a  trans¬ 
parent  aperture  of  size  /  X  w.  We  fix  «  =  rf.v/8  and 
control  the  aperture  transmittance  of  each  cell  In- 
varying  /.  To  reduce  noise  and  produce  a  good  image 
reconstruction  with  a  CGH,  we  require  dx  =  dy  « 
\fi./b ,  where  h  is  the  size  of  the  output  plane  (assumed 
to  be  square  for  simplicity).  In  our  case,  the  CGH  re¬ 
construction  or  its  impulse  response  is  a  pattern  of  delta 
functions.  The  maximum  dimension  of  this  output 
pattern  is  b  -  Ad.  For  our  CGH.  we  thus  used  d  =  1 1 
pm  and  dx  =  dy  =  ().();!  mm.  The  size  of  the  Calcomp 
plotter  pen.  the  40X  reduction  available,  and  the  reso¬ 
lution  of  the  FT  plane  film  allowed  us  to  fabricate  a 
CGH  with  only  250  X  250  cells  in  7.5  X  7.5  mm  ’.  This 
7.5-mm  FT  plane  width  will  only  pass  input  spatial 
frequencies  below  5  cycles/mm  ( for  the//  =  500- mm  FT 
lens).  Thus  input  image  detail  smaller  than  0.2  mm  will 
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The  available  aerial  image  used  |Fig.  fi(a)|  contains 
lines  li.e..  two  edges)  rather  than  one  edge  separating 
different  textured  regions.  Thus,  in  analyzing  the 
output  images  shown,  we  must  recall  that  the  edge- 
enhanced  version  of  a  line  consists  of  two  edges  of  width 
I’d  or  lid  (this  also  depends  on  the  sharpness  of  the 
edges)  separated  by  the  width  of  the  line.  The  spacing 
d  chosen  and  the  spatial  frequency  bandwidth  and 
resolution  of  the  Sol  tel  filters  also  affect  the  observed 
outputs.  A  detailed  analysis  of  the  input  image  used 
and  the  filter  parameters  chosen  confirms  that  the  re¬ 
sults  are  as  expected.  To  demonstrate  and  quantify  the 
SNR  of  the  output  images  and  the  SNR  improvement 
obtained  using  different  d  X  d  and  ">  X  f>  operators,  a 
textured  image  with  edges  (not  lines)  separating  regions 
of  different  variances  should  be  used,  '['his  typifies 
future  work  to  be  performed  on  such  processors. 

VI.  Summary  and  Conclusion 

As  image  sizes  increase,  the  nonlinear  edge-en¬ 
hancement  preprocessing  required  becomes  quite  ex¬ 
pensive  to  perform  in  real  time  by  digital  or  electronic 
methods.  The  new  techniques  we  have  developed  re¬ 
sult  in  a  coherent  optical  system  capable  of  nonlinear 
local  operators.  We  derived  and  detailed  this  technique 
for  the  case  of  the  Sobel  operator.  However,  it  can 
easily  be  extended  to  many  nonlinear  local  operators 
( l’rewitt.  Roberts,  as  well  as  the  Laplacian  and  higher- 
order  difference  operators).  We  have  described  and 
demonstrated  two  techniques  for  realization  of  such 
functions  optically.  The  multiple  MSF  technique  re¬ 
quires  a  specially  fabricated  positioning  system  for  good 
results,  but  such  a  system  is  quite  easily  produced  it 
needed.  We  demonstrated  good  ;l  X  :t  Sobel  edge- 
enhanced  output  data  on  our  initial  system  with  residts 
ilinewidths  and  SNR)  in  agreement  with  those  expected 
by  theory.  We  also  described  and  demonstrated  a  5  X 
f>  Sobel  using  C(!Hs.  The  results  were  quite  attractive 
and  again  agreed  with  theory.  A  particularly  attractive 
aspect  of  the  optical  realization  of  nonlinear  local  op¬ 
erators  is  that  larger  window  sizes  and  higher-order 
operators  can  be  used  with  no  loss  in  processing  speed. 
It  is  also  possible  to  combine  MSF.  Sobel,  and  lensless 
filters  into  one  plane.  Such  a  system  would  realize  the 
compactness  of  a  lensless  MSF  correlator  and  the 
preprocessing  provided  by  optical  Sobel  operators.  To 
obtain  the  full  benefit  from  any  such  system,  one  should 
first  determine  the  optimal  nonlinear  preprocessing 


operator  required.  The  production  of  this  operator 
with  the  desired  window  size  and  order  can  then  he 
achieved  by  parallel  lensless  nonlinear  optical  prepro¬ 
cessors  such  its  we  have  described. 
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A  variety  of  methods  have  been  proposed  for  solving  pat¬ 
tern  recognition  problems  using  optica)  methods.1  Some  of 
these  involve  generating  a  synthetic  discriminant  function 
that  is  a  weighted  combination  of  the  functions  available  as 
training  data.  The  weights  for  the  combination  are  obtained 
either  from  the  principal  component  of  the  data  set-  or  by 
imposing  different  deterministic  constraints  that  ensure  the 
separation  of  the  classes.'*  Caulfield4  has  shown  that  these 
problems  can  be  unified  as  solving  the  following  matrix-vector 
equation. 

Rw  =  c.  Hi 

where  R  is  the  matrix  of  correlation  coefficients  among  the 
members  in  the  training  set.  w  is  the  vector  of  desired  weights, 
and  c  is  the  vector  of  constraints. 

Before  we  can  solve  Eq.  (1 1  for  weight  vector  w.  we  have  to 
determine  the  correlation  coefficient  matrix  R  from  the  given 
training  data.  Various  elements  of  R  can  be  determined  op¬ 
tically’'  by  forming  cross  correlations  of  all  pairs  among  the 
training  data  set  and  then  noting  the  peak  values.  Through 
this  Letter  we  would  like  to  point  out  two  sources  of  error  in 
the  computation  of  R.  Errors  in  R  lead  to  inaccurate  weights 
which,  in  turn,  can  result  in  a  discriminant  function  that  does 
not  satisfy  the  constraints. 

A  well-known*  source  of  error  in  the  estimation  of  any  sta¬ 
tistical  parameter  is  the  finite  length  of  the  observation.  If 
the  input  functions  xU  I  and  y(t )  are  of  length  T  and  band¬ 
width  A.  we  can  easily  show*  that  the  standard  deviation  of 
the  estimated  cross-correlation  coefficient  p,,  is 

0i  *  p,,|n  +  p;,->/<2B7\„u,i|'  -.  (2i 

where  T,„ui  denotes  the  total  length  of  the  signals  used  to 
estimate p,,.  The  correlation  coefficient  p,,  is  related  to  the 
cross-correlation  function  by  normalizing  constants  that  are 
related  to  the  energies  in  the  signals  x(t )  and  ><()  We  note 
from  Eq.  (2)  that  a j  is  inversely  proportional  to  (AT,,,,,!)1 
where  (STU>U i>  denotes  the  total  time-bandwidth  product  of 
the  signal.  This  type  of  error  can  be  reduced  by  increasing 
the  length  of  observation.  In  the  case  of  images,  the  space- 
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Fig  1  Correlation  functions  for  pt,  «  0.5:  solid  line,  ideal  correla 
lion:  line  with  Xs,  observed  correlation  for  T  »  30.  line  with  Os.  ob 
served  correlation  for  T  =  100 

bandwidth  product  easily  exceeds  100(1  and  <ti  can  be  con¬ 
sidered  insignificant.  But  with  increasing  use  of  1  -D  devices 
such  as  acoustooptic  cells,  even  for  image  processing.’  care 
should  be  taken  to  see  that  this  type  of  error  is  not  signifi¬ 
cant. 

The  second  source  of  error  in  estimation  seems  to  be  less 
well  known  among  the  researchers  in  optical  pattern  recog¬ 
nition.  The  rest  of  this  Letter  will  be  devoted  to  a  discussion 
of  this  error  source.  When  two  functions  x(t  1  and  y(f  1  are 
correlated  using  matched  spatial  filter  (MSF)  techniques,  the 
nonlinear  properties  of  the  recording  medium  in  the  MSF 
plane  force  the  average  values  of  x(t )  and  y(f )  to  go  to  zero.* 
This  property  of  optical  correlators  to  force  the  mean  to  zero 
does  not  pose  a  problem  in  the  conventional  applications  of 
cross  correlators  where  the  primary  objective  is  to  locate  the 
peak  in  correlation  plane  and  the  value  of  this  peak  is  not 
important.  Then  the  effective  correlation  is  between  x’U  I 
and  y*(f  >  where 

x'tf)  *  x(/l  -  <x(f)),  >■'(!)  *  ,v(tl  -  (yin).  (3) 

and  where  (  >  denotes  the  average  over  a  duration  of  T.  Even 
in  digital  estimation  of  p,%,  we  end  up  using  only  x’(f )  and 
y'U  ),  because  the  coefficients  in  the  Fourier  transform  cor¬ 
responding  to  zero  frequencies  are  usually  suppressed. 

Let  p,y  (t)  denote  the  cross-correlation  coefficient  of  x(t  > 
andylr  +  r).  Then  p„'(0), obtained  from  the  mean  removed 
signals  x‘(t )  and  y*(t),  is  related  to  px>  (r)  as3 

P.yHO)  «  P,,(0l  -  (2/T)  J^T|1  -  (T/ri|p„(T)d7  141 

Note  from  Eq.  (41  that  we  are  underestimating  px>  (0)  be¬ 
cause  of  the  use  of  mean  removed  signals  x'U  1  and  y’U  1.  To 
better  understand  the  magnitude  of  this  error,  let  us  assume 
px>  (r)  to  be  an  exponential  function 

Pi»(r I  “  Pol r I  “  exp|l— o  |r|  I],  (5) 

where  p„  is  inversely  related  to  the  bandwidth  of  the  signal 
and  a  is  a  measure  of  the  signal  bandwidth.  For  this  corre¬ 
lation  function  we  can  show  that 


i  '  f-  -  :  •  • ;  • :  4  ■ ;  t  ’  L 


Fig  2  Correlation  functions  for  p„  *  0.8:  aolid  line,  ideal  correla 
tion;  line  with  Xs.  observed  correlation  for  T  «  30:  line  with  Os.  ob¬ 
served  correlation  for  T  =  100 

p,,M(U  -  1  -  (2/aT i  +  (2/o-T-i  -  l2/a-T-i  expi -cT >  ifii 

For  large  (qD  values,  the  above  relation  can  be  approxi¬ 
mated  as 

P,,'|0)  =  1  -  (2,o7'i  (7 1 

Thus  the  removal  of  mean  from  a  signal  xU )  can  cause  a 
significant  bias  in  the  estimated  value  of  pt>  (0).  This  can  be 
a  significant  problem  when  we  use  low  time-bandwidth  de¬ 
vices  to  process  long  duration  signals  or  large  images  Since 
these  devices  can  accommodate  only  a  limited  time-band 
width  product  at  any  instant,  mean  removal  can  cause  a  sig- 
nificant  bias  in  px>  (0)  estimated  from  each  segment. 

To  illustrate  the  above  problems,  we  generated  digital  se¬ 
quences  of  lengths  T  -  30  and  T  =  100  with  pv  values  of  0.5 
and  0.8.  Ideal  autocorrelation  functions  according  to  Eq  <5i 
and  the  autocorrelation  functions  estimated  from  the  mean 
removed  signals  are  shown  in  Figs.  1  and  2.  The  results  in 
these  figures  were  obtained  by  averaging  over  fifty  indepen 
dent  seq uences.  Thus  the  random  error  in  Eq.  <  2 1  result  i ng 
from  the  finite  signal  length  should  be  negligibly  small  But 
the  bias  errors  due  to  the  mean  removal  do  not  average  out  and 
can  be  clearly  seen  in  Figs.  1  and  2.  In  Fig.  1,  p  is  0.8  (small 
TBW)  and  biases  are  large,  whereas  in  Fig.  2.  p  is  0.5  (large 
TBW)  and  biases  are  small.  In  fact,  the  expected  values  for 
pa, MO)  according  to  Eq.  (7)  are  0.972  and  0.911  for  T  =  100. 
whereas  we  observe  0.964  and  0.937  experimentally.  For  T 
-  30,  theoretically  expected  values  are  0.904  and  0.708. 
whereas  the  experimentally  obtained  values  are  0.900  and 
0.775.  As  can  be  seen,  the  experimental  results  are  close  to 
the  theoretically  expected  values. 

We  note  from  our  results  that  the  correlation  coefficients 
obtained  from  optical  MSF  correlators  should  be  accepted 
only  if  the  time-bandwidth  products  of  the  signals  are  large. 
Otherwise,  the  biases  in  them  should  be  corrected  by  adding 
(2/oD  where  (oT)  is  the  time-bandwidth  product.  Often  we 
do  not  know  the  a  priori  value  of  a  and  this  must  also  be  es 
timated.  This  causes  variance  in  estimated  correlation 
coefficients  and  one  has  to  take  this  into  account 
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11.  CORRELATION  OF  BINARIZED  IMAGES 


Correlation  of  Binarized  Images 

Correlator  technique  has  been  used  in  pattern  recognition  appli¬ 
cations.  We  introduce  a  correlation  performance  measure,  peak  to 
background  ratio  (PBR),  defined  as  the  ratio  of  the  peak  intensity 
to  the  average  background  variation  in  a  single  correlation  output. 
Using  this  performance  measure,  we  analyze  the  binary  correlator 
and  show  that  the  binarization  of  the  images  can  result  in  better 
performance.  We  consider  a  real-time  implementation  of  the  binary 
correlator,  using  VLSI  technology.  The  analytical  results  are  sup¬ 
ported  by  computer  simulations. 

I.  INTRODUCTION 

Matched  filter  or  correlator  recover  was  originally  1 1 ) 
derived  as  the  optimum  linear  time-invariant  filler  to  dc- 
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tect  a  known  signal  in  the  presence  of  additive  white 
noise  The  performance  criterion  used  in  this  situation 
was  the  ratio  of  the  average  value  of  the  correlation  out¬ 
put  at  a  particular  time  instant  to  the  standard  deviation  of 
the  output  at  that  time  instant.  This  ratio  is  common!) 
known  as  the  signal-to  noise  ratio  (SNR)  and  has  been 
the  optimality  criterion  in  many  communication  receiver 
designs. 

Recently,  correlation  operation  has  found  widespread 
application  in  image  pattern  recognition  because  of  ad¬ 
vances  in  coherent  optical  methods  |2|  as  well  as  digital 
methods  1 3 1 .  In  such  image  correlations,  we  are  interested 
not  only  in  detecting  the  presence  of  a  "target"  in  a 
noisy  "scene."  hut  also  in  estimating  the  location  of  the 
target  in  the  scene.  For  this  purpose,  it  is  desirable  to 
keep  the  correlation  peak  (at  the  true  location)  as  high  as 
possible  while  reducing  the  sidelobe  levels  all  around  this 
peak.  As  a  result,  the  ratio  of  average  correlation  peak  to 
the  standard  deviation  of  output  correlation  "far”  from 
the  peak,  commonly  denoted  as  peak-to-sidelobe  ratio 
(PSR).  is  used  |4|  as  a  performance  measure  to  evaluate 
the  image  correlations.  PSR  has  been  widely  used  to 
evaluate  image  correlations  in  the  presence  of  image  dis¬ 
tortions  such  as  coordinate  transforms  |4.  5|.  frequency 
plane  blurring  |6|.  and  input  transducer  nonlinearity  in 
coherent  optical  correlators  |7| 

The  arbitrary  nature  of  sidelobe  determination  in  PSR 
calculations  is  evident  in  the  selection  of  position  "far" 
from  the  peak.  In  fact,  a  recent  paper  |K|  talks  about  the 
PSR  at  three  different  locations:  exact  registration,  near 
misregistration,  and  gross  misregistration.  We  feel  that  a 
good  measure  of  the  correlator  performance  should  utilize 
the  variance  at  all  points  in  the  correlation  plane  rather 
than  at  any  single  point.  This  will  avoid  the  possibility  of 
getting  a  high  PSR  simply  because  we  happen  to  choose 
a  correlation  plane  location  where  variance  was  small.  In 
this  paper,  we  utilize  a  somewhat  similar  measure  known 
as  peak-to-background  ratio  (PBR).  PBR  is  defined  as  the 
ratio  of  the  average  correlation  peak  to  the  average  ol 
standard  deviations  at  all  points  in  the  output  One  can 
generalize  the  PBR  by  appropriately  weighting  the  stan¬ 
dard  deviations  in  the  correlation  output  before  averaging 
them.  For  example,  standard  deviations  of  points  close  to 
registration  may  be  given  more  importance  compared 
with  points  of  gross  misregistration. 

We  analyze  the  performance  of  the  binary  correlator 
in  this  paper  using  PBR.  The  conventional  correlator  per¬ 
forms  poorly  in  comparison  to  some  sequential  similarity 
deleetit m  algorithms  tSSDA)  |4|  in  terms  of  computa¬ 
tional  efficiency.  Such  computational  considerations  have 
led  to  renewed  interest  in  the  use  of  correlators  where  the 
input  signals  are  hard  limited  (to  two  amplitude  levels) 
prior  to  correlation.  Svedlow  et  al.  1 1()|  report  the  experi¬ 
mental  use  of  such  binary  correlators.  Vijaya  Kumar  1 1 1 1 
analyzes  the  performance  using  PSR,  Boland  et  al  1 1 2 1 
suggest  a  real-time  digital  implementation,  while  Cole 
1 1 3 1  proposes  an  incoherent  optical  processor  for  binary 
correlation  In  Section  II.  wc  show  that  the  binarizalion 


does  not  appreciably  degrade  the  PBR  of  a  correlator 
Our  theoretical  results  are  in  good  agreement  with  the 
prior  work  1 1  1 1  utilizing  PSR  as  the  criterion  We  present 
simulation  results  in  Section  III  to  support  the  results  of 
our  analysis.  The  main  advantage  of  a  binary  correlator  is 
that  it  can  be  realized  w  ithout  multiplication  It  is  thus 
ideally  suited  for  direct  digital  logic  design  and  we  con¬ 
sider  an  implementation  of  this  using  VLSI  technology 
1 1 4 1  in  Section  IV 


II.  BINARY  CORRELATOR  PERFORMANCE 

Let  ft,}  and  {y,}.  i  =  1.2 . V  denote  the  target  and 

scene,  respectively  .  One-dimensional  notation  is  used 
only  for  simplicity  and  extension  to  the  two-dimensional 
case  of  images  is  straightforward.  In  general.  is  related 
to  by 

v,  v,  +■  n,  (I) 

where  /„  denotes  the  shit t  between  the  two  signals  and  n, 
is  a  sequence  ol  zero  mean  uncorrelated  random  varia¬ 
bles.  Without  any  loss  of  generality,  we  assume  to  be 
zero  in  the  rest  ol  this  paper.  Then  the  correlation  output 
is  given  as 

\ 

C(/)  -  (I  /V )  ^  A  (/))(/  -  /).  /  -  1.2 . ;V  (2) 

I  I 

where  we  have  implicitly  assumed  that  y(/l  is  periodically 
repeated  lor  values  of  arguments  beyond  ,V.  This  is  pre¬ 
cisely  what  happens  w  hen  correlations  are  performed  dig¬ 
itally  using  discreet  Fourier  transforms  1 1 5 1  For  this  cor¬ 
relation  output  Cl/),  we  define  the  PBR  y  as  follows: 

\ 

y  -  vic«»|j  <  v  |(-(/,|-  >  (3) 

:  I 

is  the  ratio  of  the  correlation  peak  intensity  (when  the  ref¬ 
erence  and  input  are  perfectly  matched)  to  the  average 
correlation  output  over  the  entire  range  (not  just  at  /  0 

or  i  ■  0)  C(/l  is  m  general  very  small  as  /  increases. 
Thus  wc  note  from  (3)  that  PBR  increases  as  the  length  V 
ol  the  signals  increases.  T  his  supports  the  experimental 
observation  |  lb|  that  the  PBR  of  a  correlator  is  directly 
proportional  to  the  space  bandwidth  product  ol  the  input 
data 

We  evaluate  the  performance  of  the  binary  correlator 
in  this  section  The  input  signals  {.y(  and  jv  I  are  sub¬ 
jected  to  hard  limiting  be  I  ore  correlation: 


where  the  caret  denotes  binarized  variables  Wc  have  im 
plicitly  assumed  above  that  the  input  signals  (  and  |  v, } 
arc  both  ol  zero  mean  Such  an  assumption  is  |usiified 
due  to  one  ol  the  following  two  reasons  In  coherent  opti 

mix  ‘>751  xr  (lino  ih:<  ym  ?s  .  iwi  II  I  l 
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cal  processors  |16).  the  matched  filter  plane  medium  usu¬ 
ally  exhibits  a  dark  central  region,  thus  removing  any 
average  value  present  in  the  input  signals.  Average  values 
in  images  are  deliberately  removed  |3]  in  many  image 
correlations  because  it  results  in  an  output  bias  level,  thus 
degrading  the  output  correlation  PBR 

The  binary  correlation  peak  value  in  the  absence  of 
noise  is  given  by 
,v 

C< 0)  =  <IW)|  2  =  a2  (5) 

i-  i 

which  can  be  seen  to  be  the  maximum  possible  peak 
value  for  a  given  input  dynamic  range  |  -  a.  +  «|.  In 
digital  correlators,  correlation  peak  values  may  not  be  im¬ 
portant.  but  in  optical  correlators,  system  noises  such  as 
detector  noise,  reflections,  speckle  noise,  and  scatter  indi¬ 
cate  the  need  for  a  large  correlation  peak  1 1 1  j.  Thus  bi¬ 
nary  correlation  reduces  the  effects  of  system  noise  in 
optical  correlators. 

PBR  of  binary  correlation  can  be  written  as 
.v  iV 

y  =  N(a*)i  X  |(  MN)  X  VC*,)2  (6) 

;-i  <‘i 

where  we  have  assumed  no  noise  in  the  input.  We  con¬ 
sider  the  effect  of  noise  at  a  later  stage  in  this  section. 

It  is  difficult  to  evaluate  the  denominator  of  (6)  unless 
the  input  signal  sequence  {x,}  is  known.  The  exact  nature 
of  {.r,}  is  rarely  known  a  priori,  but  we  will  assume  that 
{x}  is  a  sample  realization  from  a  zero  mean  Gaussian 
process  with  a  second-order  statistics  R.L/j-  With  such  an 
assumption,  we  have 

£{x, -*,♦,}  =  <r{Pr(x,.r, ,,  >  0|  -Pr|.r,.r,  v,  s  0)1 

=  «:{(2/tt)  sin  l(^,(J)//f,(0)|}  (7) 


where  £{•}  denotes  ensemble  average  and  the  last  equal¬ 
ity  is  derived  elsewhere  |I7).  The  quantity  within  the  par¬ 
enthesis  in  the  denominator  of  (6)  can  be  seen  to  be  a 
sample  estimate  of  the  ensemble  average  in  (7).  For  suffi¬ 
ciently  large  N.  the  error  in  the  sample  estimate  is  ex¬ 
pected  to  be  small  and  we  can  approximate  y  as 
v 

*Y  -  N  «4/  2  (£(*,*,♦,) I2 

f-  \ 

N 

=  An1/  X  {(2/ir)«:  sin  '\R,[j)lRkmV 

JV 

=  tV(ir/2)2/|  X  <sin  m 

i 

One  can  show  that  for  any  0  s  P  5  1 , 

P  s  sin  1  P  ^  fir/2)  P  (9) 

and  using  this  inequality  in  (8).  we  can  bound  y  as 
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/VlR.2(0)/  X  *.20')l  s  y  s  A/| tt/2 ) 2 

./-  I 

s 

•  |/?,2(0)/  X  K.20)l  (10) 

I- 1 

We  can  calculate  the  expected  PBR  for  any  input  sig¬ 
nal  by  substituting  its  autocorrelation  function  R,(j)  in  (8) 
and  evaluating  it.  To  understand  the  effects  of  binariza- 
tion  on  PBR.  we  estimate  the  PBR  y  of  unbinarized  cor¬ 
relation  from  (3)  using  a  similar  approach  as 


y  =  N\R2(0)l  X  f(,20)l  (ID 

i- i 

Comparing  ( 10)  and  (II),  we  note  that  PBR  y  for  the 
binary  correlation  is  greater  than  the  PBR  y  for  the  unbi- 
narized  correlation.  We  also  can  see  that  this  improve¬ 
ment  is  at  the  most  a  factor  of  < 'rr/2)2  or  3.8  dB  The 
analysis  of  the  binary  correlator  has  so  far  assumed  no 
noise  at  the  input.  To  analyze  the  effect  of  noise,  we  re¬ 
place  the  signal  {x,}  by  {x,}  +  {«,},  where  {«,}  is  a  sam¬ 
ple  realization  from  a  zero  mean.  Gaussian  random 
process  with  an  autocorrelation  function  R„(jl.  {xj  and 
{n,}  are  also  assumed  to  be  jointly  Gaussian  and 
uncorrelated. 

Proceeding  as  before,  the  correlation  peak  value  for 
the  noisy  case  is  given  by 

C( 0)  =  (1//V)  X  x,<-x,  +  «,) 

i-  i 

s  (2/7 r)a2  sin  'j|  R,(0)|/ 

V<f,(0)|R,(0)  "+  R„(0)|  } 

=  (2/TT)rr  sin  '  \/R,<0)/|R,(0)  +  /?„(()) |  (12) 


Comparing  (12)  with  (5),  we  note  that  the  correlation 
peak  value  decreases  as  the  amount  of  input  noise  /?„(()) 
increases.  The  PBR  y  in  the  presence  of  the  noise  can  be 
derived  as 


y  =  {(IAV)  X  +  «,>J2 


N 

1 

I  —  1 

/V 


•{(1/A0  X  l(,/Af)  X  X,(JC1(,  +  «,,, 

i ~  i  <-i 


=  1(2/tt)o2  sin  1  VR,(0)/{Rt<0)  +  R„(0)}|: 

*  {(l/jV)(2a2/ir)2 

N  _ _ 

•  X  Isin  1  VrTwWW)  +  /?,(0>/?n<0)}|2}  1 

j‘  i 

=  A|  sin  1  VR,(0)/{R,(0)  +  R„(0)}|2 
•{  X  ls»n  1  VR,f(j)/{R?(0)  +  /?,(0)R„<0)}|2}  1 

I  - 1 


(13) 
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Using  the  inequality  of  (V)  in  (13).  we  can  show  that 
( 2/tt) '  y  -  y  —  (tr/2):  y  1 13) 

where  y  is  the  PBR  of  unbinarized  correlation  as  in  ( 1 1 ). 

The  inequality  in  (14)  shows  that  the  ratio  (yly)  var¬ 
ies  from  -  3.8  dB  to  +  3.8  dB;  thus  depending  on  the 
input  noise  level,  we  will  find  situations  where  the  binary 
correlator  will  outperform  the  unbinarized  correlator  As 
we  have  shown  in  (8).  Iy  is  closer  to  the  upperbound  for 
the  case  of  low  noise.  It  has  been  shown  elsewhere  1 1 1 1 
using  peak  to  sidelobe  ratio  (PSR)  that  binarization  results 
in  a  loss  of  3.8  dB  in  the  case  of  very  noisy  inputs.  Us¬ 
ing  an  exponential  model  for  R,(j).  Kumar  and  Casasent 
1 1 1 1  examine  the  PSR  for  binarized  correlation  as  a  func¬ 
tion  of  signal  bandwidth. 

In  this  section,  we  have  shown  that  binary  correlators 
perform  at  least  as  well  as  unbinarized  correlators.  But 
their  main  advantage  is  in  the  ease  of  implementation 
which  is  considered  in  Section  IV  Before  we  examine 


PI 


the  simulation  results,  a  few  words  of  caution  are  in  or¬ 
der.  We  considered  the  performance  only  from  the  view¬ 
point  of  PBR.  but  as  the  input  noise  level  increases,  there 
is  a  greater  probability  that  the  correlation  has  a  maxi¬ 
mum  at  j  f  0.  The  effect  of  input  binarization  on  this 
false  peak  probability  is  of  importance  in  location  estima¬ 
tion  problems  Correlation  is  often  used  to  separate  two 
classes  of  targets.  Binarization  effects  on  class  discrimi¬ 
nation  should  also  be  considered  When  we  use  the  defi¬ 
nition  in  (3)  for  PBR.  we  are  implicitly  assuming  that  the 
correlation  has  a  peak  at  /  -  0.  This  is  true  only  for  low 
noise  situations  and  thus  our  conclusions  should  not  be 
used  for  high  noise  input  scenarios. 

III.  SIMULATION  RESULTS 

In  this  section,  we  support  the  conclusions  of  Section 
II  with  the  help  of  a  very  limited  image  data  set  shown  in 
Fig.  I.  These  are  all  digitized  images  of  the  same  area 


P2 
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rather  than  the  mean  to  satisfy  our  assumption  that  the 
average  value  of  {a,}  is  zero.  Even  though  the  gray  levels 
as  shown  in  Fig.  2  represent  completely  dark  (0)  and 
completely  white  (  +  1)  areas,  we  treat  them  as  denoting 
-  1  and  +  I  in  our  actual  processing.  Cross  correlations 
are  performed  on  these  binary  images  and  the  resulting 
parameters  are  shown  in  Table  II.  Once  again  S,  corre- 

TABI.I:  11 

Performance  of  Biiun/ed  Correlation 


s,  s,  s,  s. 


A  Normalized  Intensities  at  Cross-Correlation  Origins 


s, 

1.0000 

0  5806 

02171 

02101 

S, 

0  5806 

1  0000 

02157 

0.1911 

S, 

0.2171 

02157 

1 .0000 

0.6733 

S4 

02101 

0. 191 1 

0  6733 

1  00(X) 

B  Peak  to 

Background 

Ratios  (PBRs) 

s, 

10027 

64  595 

44.880 

48.693 

s. 

64.595 

117.48 

49.385 

49  507 

s, 

44.879 

49.385 

139.56 

100  51 

s4 

48.693 

49.507 

100  51 

154.00 

lates  better  with  S2  than  with  S,  or  S4.  Comparing  the  di¬ 
agonal  elements  of  Table  IB  and  Table  IIB.  we  see  that 
unbinarized  autocorrelations  yield  PBRs  of  63.2,  43.6, 
44.4,  and  76.7.  while  binarized  autocorrelations  yield 
PBRs  of  100.3.  117.5.  139.6.  and  154.0,  respectively. 
Thus  we  see  that  for  all  four  images,  binarization  im¬ 
proves  the  correlation  PBR  by  factors  of  1.25  to  2.0. 
These  improvement  factors  are  well  within  the  range  of 
1.0  to  lit  2)2  =  2.5  predicted  by  ( 10).  The  PBRs  in  all 
cases  are  estimated  using  the  definition  in  (3)  on  the  ob¬ 
served  correlation  output  The  PBRs  for  binarized  cross 
correlations  are  consistently  higher  than  the  PBRs  for  un¬ 
binarized  correlations.  Improvement  factors  of  almost  5.0 
(beyond  the  2.5  suggested  by  (10))  are  observed  in  the 
cross  correlations  of  S,  and  S4  and  P,  and  P4.  This  is  be¬ 
cause  P|  and  P4  differ  drastically  (through  random  con¬ 
trast  reversals)  and  the  differences  cannot  be  modeled 
simply  as  additive  noise  Binarizing  removes  the  unrelia¬ 
ble  amplitude  information  while  retaining  only  the  major 
edge  information  and  thus  results  in  better  PBR  ( 1 8 1 . 

We  now  observe  the  effects  of  additive  noise  in  the 
input  on  the  observed  autocorrelations.  In  Table  III.  we 

TABI.I;  III 

Effect  of  Input  White  Noise  on  L'nhinarized  Correlations 


\  S 

Peak 

PBR 

0  00 

1  0000 

63  163 

002 

0  9805 

63  (153 

0  04 

0  9621 

63  548 

0  06 

09439 

63.759 

0  OK 

0  9267 

63  365 

0  H> 

0  9095 

63  144 

0  12 

0  K957 

63  5(15 

0  14 

0  KK09 

61  836 

0  16 

0  K644 

61  450 

0  IK 

0  KSW 

63  3.33 

Notes  Inpul  is  (he  intensity 

normalized  {P, 

*  noise)  Relerence  is 

(he  rntcnMf)  normalized  fP,} 


show  the  peaks  and  PBRs  obtained  as  different  amounts 
of  additive  white  noise  is  introduced  in  the  autocorrela¬ 
tion  P,  *  P|.  The  ratio  NlS  indicates  the  white  noise 
power  to  the  image  (signal)  power.  The  peak  values  and 
PBRs  obtained  in  the  binarized  autocorrelation  S ,  *  S ,  for 
various  N/S  values  are  shown  in  Table  IV.  For  the  rea- 

TABLE  IV 


Effect  of  Inpul  While  Noise  on  B  man  zed  Correlation 


MS 

Peak 

PBR 

0.00 

1  0000 

100  37 

0  03 

0  9139 

93  301 

0.04 

0  8707 

89  097 

0  06 

0.8403 

86  798 

0  08 

0.8175 

86.640 

0  10 

0.7848 

83.453 

0  12 

0  7680 

83  397 

0.14 

0.7473 

81270 

0  16 

0  7333 

80  563 

0.18 

0  7308 

81  689 

Notes:  Input  is  ihe  binarized  {P|  +  noise}  Reference  is  the  binar- 
i7.ed  (Pi) 


sons  discussed  in  Section  II.  only  small  amounts  of  noise 
are  considered  here.  From  Table  III.  we  note  that  PBR  of 
P,  *  P,  is  relatively  unaffected  by  small  amounts  of 
noise.  On  the  other  hand,  the  PBR  of  binarized  correla¬ 
tion  decreases  as  noise  increases;  but  for  the  range  of 
noise  levels  shown,  binarized  correlation  still  gives  better 
PBR  compared  with  unbinarized  correlation.  Binarized 
correlation  peak  is  also  seen  to  be  more  sensitive  to  noise 
in  the  input  signal.  Thus  we  see  that  binary  correlator  still 
outperforms  unbinarized  correlation  in  the  presence  of 
small  amounts  of  noise. 

IV.  DISCUSSION 

In  this  paper  we  have  shown  that  binary  correlation 
gives  better  peak-to-baekground  ratios  even  in  the  pres¬ 
ence  of  small  amounts  of  noise.  We  supported  these  theo¬ 
retical  arguments  w  ith  the  help  of  simulation  on  a  set  of 
mullispcctral  images. 

The  binary  correlator  is  attractive  mainly  from  itnple- 
mcntational  considerations  By  using  the  VLSI  technol¬ 
ogy  1 14],  the  binary  correlator  can  be  implemented  in  a 
reasonable  number  of  chips.  The  ♦  a  and  -a  levels  of 
the  binarized  signal  in  (4)  correspond  to  logical  I  and  0 
in  a  digital  implementation.  The  multiplication  laws  are 
then  as  given  in  Table  V.  We  observe  from  this  table  that 
TABI.I.  v 

Multiplication  l  aws  for  Binary  Input- 

x  y  xt  i  |.vi  l  | yj  /.  |X)  | 

11  a  o'  I  I  I 

a  a  -«■’  I  (l  ll 

o  a  -o-  (I  I  (I 

<i  <i  u'  II  0  I 

Volt  /.  |  |  refers  to  the  corresponding  logic  level 
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a  binary  multiplication  can  be  easily  implemented  by  a 
simple  exclusive-OR  circuit.  In  order  to  obtain  the  binary 

correlation  Cij).  we  must  add  (  t,v, ,;)  for  all  values  of  / 
for  every  j.  For  a  sequence  of  length  N.  let  M ,  represent 
the  number  of  the  times  the  product  (  v,v, . ,)  equals  +  ir 
tor  +  1  in  digital  circuits).  Then  ('( j)  can  be  written  as 

Cij)  =  <rMt  -  ir{.\  -  M, ) 

=  u:(2M,  -  .Vi.  (15) 

Thus  Cij)  can  be  easily  obtained  by  just  counting  the 
number  of  times  the  product  (i,v, equals  +  I  among 
all  possible  /  for  a  fixed  j.  This  can  be  implemented  as  in 
Fig.  3.  In  Fig.  3.  the  reference  sequence  {.i,}  is  stored  in 


Fig  3.  Digital  binary  correlator  circuit  R  is  a  I -bit  shift  register.  +  is 
an  exclusive-OR  circuit,  and  Mf  is  the  counter  input 


the  top  array  of  shift  registers  and  then  by  serially  shift¬ 
ing  {>',}  through  the  second  array,  the  serial  output  { M t\  is 
obtained.  C(j)  can  be  obtained  from  Mt  using  (15)  in  a 
postprocessor. 
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Generalized  chord  transformation  for  distortion-invariant 
optical  pattern  recognition 


David  Casasent  and  Wen-Thong  Chang 


An  optical  processor  that  realizes  a  generalized  chord  transformation  is  described.  The  wedge-ring  detector 
samples  of  an  autocorrelation  are  she  vn  to  he  the  histograms  of  t  lie  chord  distributions.  This  dimensionali¬ 
ty  reduced  set  of  features  is  used  as  the  feature  vector  inputs  tor  a  Fisher  linear  classifier  to  determine  the 
class  of  the  input  object  independent  of  geometrical  distortions.  Initial  discussions  on  the  use  ot  different 
classifiers,  the  polarity  of  the  classifier's  output,  and  selection  of  the  image  training  set  are  also  advanced 


I.  Introduction 

Obtaining  object  classification  in  the  face  of  geo¬ 
metrical  distortions  in  the  input  object  (due  to  scale, 
rotation,  aspect  variations,  etc.)  is  a  major  pattern  rec¬ 
ognition  problem  that  has  received  extensive  attention. 
Various  optical  systems  of  increasing  flexibility  and 
potential  have  been  suggested  as  partial  solutions  to  this 
problem.1  These  include  weighted  matched  spatial 
filters.-  space-variant  optical  pattern  recognition  sys¬ 
tems,1  optical  correlators  using  generalized  matched 
spatial  filters.1  and  synthetic  discriminant  functions.'1 
Conventional  digital  techniques  for  distortion-invariant 
pattern  recognition  involve  extracting  features  from 
regions  of  the  input  image,  determination  of  a  linear 
discriminant  function  or  linear  combination  of  these 
features  (from  image  training  sets  of  the  different  object 
classes),  and  using  the  measured  features  for  an  input 
object  in  a  classifier  to  determine  the  class  of  the  input 
object.  A  recent  optical  system*5  follows  this  approach 
by  extracting  the  moments  of  the  input  object  optically 
and  then  using  these  as  the  features  to  be  fed  to  a  digital 
nonlinear  least -mean-square  estimator  to  perform 
object  classification. 

In  this  paper,  we  describe  a  new  optical  architecture 
that  realizes  a  new  operation,  a  generalized  chord 
transformation  (Sec.  ID.  This  constitutes  our  obser¬ 
vation  space.  We  utilize  a  wedge-ring  detector1  to  ex¬ 
tract  features  from  this  output  for  reduced  dimen¬ 
sionality  and  for  distortion-invariant  pattern  recogni¬ 
tion  (Sec.  III).  These  measured  features  are  then  fed 
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to  a  Fisher  classifier*  (Sec.  IV)  from  which  the  class  of 
the  input  object  is  determined  even  if  geometrical  dis¬ 
tortions  are  present  in  the  input  object.  Initial  simu¬ 
lation  results  (Sec.  V)  are  most  impressive.  Extensions 
to  multiclass  problems  and  the  use  of  alternate  classi¬ 
fiers  are  then  advanced  together  with  new  techniques 
to  select  the  training  set  image  data  (Sec.  VI). 


II.  Optical  Realization  of  a  Generalized  Chord 
Transformation 

The  chord  transformation**  is  defined  for  a  binary 
solid  object  described  only  by  its  boundary  li.e..  Fig.  1 ). 
For  each  pair  of  points  on  the  boundary,  we  construct 
a  chord  and  note  its  length  r  and  angle  I).  We  then  de¬ 
scribe  the  object  (boundary)  by  the  distribution  lUr.  Wl 
of  all  chords.  This  conversion  of  a  binary  solid  object 
f(x,y)  into  h(r.O)  is  a  chord  transformation. 

To  develop  a  generalized  chord  transformation  and 
its  optical  realization,  we  first  consider  a  simplified 
technique  (suitable  for  optical  realization)  to  produce 
the  chord  transformation.  We  consider  an  image  and 
denote  points  in  the  image  by  (.v.y)  and  the  boundary 
of  the  image  by  those  points  btx.y )  =1.  A  chord  then 
exists  between  two  points  if 

.4 =  h\.\,\)b{.\  +  r  vos  +  r  sin/M  =1.  ill 

i.e.,  if  the  two  points  are  on  the  boundary.  =  1 .  and  a 
chord  defined  by  r  and  0  exists.  The  distribution  htrJD 
of  chords  in  the  image  can  then  be  described  from 

h'irji  I  =  .1.1  gt  \\\\rJhdxd\ 

=  .1.1  /»(.v.y  )/»l.v  +  /■  vns^.r  +  r  sin^)</w/\  (2l 

Substituting  (£.>?)  =  (r  cos ll.r  shift).  Kq.  (2)  becomes 

/it£.»ji  =  .1.1  hix.\  \h  i  -t  £.\  +  f/l( y.u/\ 

=  hi\ .\  I  ■*  /Hr  ,y  I.  i 
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which  we  recognize  as  the  autocorrelation  of  the 
boundary  b  l.r.y) 

'I’he  calculation  of  all  (r.O)  values  for  all  boundary 
points  is  very  time-consuming.  However,  from  Eq.  (3). 
we  see  that  the  /i(£.>;)  distribution  can  also  be  obtained 
by  simply  autocorrelation  the  boundary  of  the  object. 
(This  fact  was  also  noted  in  Ref.  10  and  elsewhere.) 
Since  optical  processors  easily  realize  the  correlation 
operation,  an  optical  realization  of  the  chord  transform 
appears  quite  attractive.  However,  optical  systems  can 
realize  the  autocorrelation  of  the  full  object  (not  just  its 
boundary)  with  no  increase  in  system  complexity.  We 
denote  the  autocorrelation  of  the  full  image  f(x,y)  rather 
than  just  its  boundary  6(.v,y)  by 

tu, =  \\fix.y ■i/’i.r  +■  £.y  +  r\\dxd\  =  f ’  •  (.  Ui 

Since  this  autocorrelation  is  not  restricted  to  the 
boundary  of  the  object  nor  is  it  restricted  to  binary- 
objects.  we  denote  the  result  as  a  generalized  chord 
transformation. 

One  can  optically  realize  the  autocorrelation  of  an 
image  by  several  techniques.  Nichols11  used  a  two- 
stage  Fourier  transform  in  which  the  autocorrelation 
w-as  obtained  from  the  Fourier  transform  of  the  power 
spectrum.  If  two  2-D  spatial  light  modulators  are  vised, 
the  autocorrelation  can  be  realized  entirely  optically  in 
parallel  on  one  system.  A  joint  transform  correlator1- 
using  mirrors  or  point  holographic  elements  to  replicate 
the  input  pattern  is  yet  another  possibility.  With  a 
nonlinear  crystal  positioned  in  the  joint  Fourier  trans¬ 
form  plane,  these  systems  can  be  further  simplified. 
Thus  the  optical  realization  of  the  autocorrelation  of  an 
input  image  (as  required  to  produce  a  chord  transfor¬ 
mation)  is  easily  achieved  by  many  techniques. 


computed  from  the  optical  autocorrelation,  and  dif¬ 
ferences  in  both  distributions  were  noted  to  exist  for 
different  letters  of  the  alphabet.  However,  no  classi¬ 
fication  algorithm  was  suggested.  In  Ref.  Ft.  the  sum 
and  difference  of  the  angles  between  the  chords  and  the 
boundary  of  the  object  were  calculated.  The  authors 
refer  to  the  distribution  of  these  angular  sums  and  dif¬ 
ferences  as  a  .(-I)  histogram.  In  Ref.  1  3,  such  a  display 
was  considered  for  ship  classification.  In  Ref.  1-1,  the 
distribution  of  the  lengths  of  the  chords  was  digitally 
computed  for  various  noisy  circular  objects  and  map 
outlines. 

We  found  much  potential  shape  information  and 
pattern  recognition  potential  in  the  chord  transforma¬ 
tion.  However,  the  computational  complexity  required 
in  prior  techniques,  the  use  of  only  boundary  objects  in 
prior  work,  and  the  lack  of  attention  to  the  classification 
algorithm  to  use  motivated  our  present  work.  We  first 
consider  a  new  technique  to  extract  the  h(r.ll)  radial  and 
angular  chord  distributions  from  an  optically  generated 
autocorrelation  of  the  input  image.  As  noted  earlier, 
our  h,;(rj>)  will  be  the  distribution  of  a  generalized 
chord  transformation.  Hereafter,  we  do  not  specifically 
distinguish  between  h  and  h,;. 

As  a  preferable  approach,  we  consider  a  simple  and 
direct  technique  for  feature  extraction  from  ihe  gen¬ 
eralized  chord  transformation  observation  space  with 
attention  to  dimensionality  reduction  and  obtaining 
distortion-invariant  features.  Dimensionality  reduc¬ 
tion  is  essential,  since  otherwise  /)(£,t?)  can  contain  l()fi 
output  samples  (for  a  10:!  X  10:,-pixel  image),  and  the 
corresponding  object  would  have  to  be  modeled  as  a 
vector  f  with  dimensionality  D  =  10tt.  Processing  such 
vector  data  would  be  computationally  intense.  An  at¬ 
tractive,  easily  realized,  and  intuitively  pleasing  choice 
for  a  feature  space  results  if  the  h  (£.t;)  optical  autocor¬ 
relation  output  plane  pattern  is  sampled  with  a 
wedge-ring  detector.  This  detector*  consists  of  32 
wedge-shaped  detector  elements  in  one-half  of  a 
2.54-cm  ( 1  -in. )  circular  detector  and  32  annular  or 
ring-shaped  detector  elements  in  the  other  half  of  the 
circular  detector  area.  All  64  outputs  are  available  in 
parallel  on  64  separate  leads.  Figure  2  shows  the  gen¬ 
eral  schematic  of  our  hybrid  processor.  We  discuss  the 
features  produced  below,  and  in  Sec.  IV  we  discuss  the 
classifier. 

Denoting  our  generalized  chord  transformation  ob¬ 
servation  space  by  /)(£.)/)  =  h(r  cos (l,r  sinW),  where  £  = 
r  cosft  and  ti  =  r  sinO.  the  wedge-sampled  outputs  are 


III.  Feature  Space 

Much  research1"*1 11  Ul  has  been  performed  on  dif¬ 
ferent  manipulations  and  displays  of  the  chord  trans¬ 
formation  output  distributions  to  achieve  distortion- 
invariant  pattern  recognition.  In  Ref.  10.  the  distri¬ 
butions  of  the  lengths  and  angles  of  the  chords  of  an 
object  were  considered  as  features.  However,  only  the 
peak  value  of  the  distribution  was  used,  and  the  classi¬ 
fication  of  only  several  simple  geometrical  shapes  was 
considered.  In  Ref.  1 1.  the  length  and  angle  distribu¬ 
tion  of  the  chords  of  different  characters  were  digitally 


MM  -  J  Mr  cosMr  sinMrtfr.  (5a) 

and  the  ring-sampled  detector  outputs  are 

Mr)  =  )  Mr  cosfl.r  sinMrdM,  (5b) 


Fiji-  2.  Simplified  block  diagram  of  a  hybrid  optical/digital  distor- 
lion  invariant  pattern  recognition  system  using  a  generalized  chord 
transformation. 
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Inspection  of  Kq.  (5a)  shows  that  the  h  {0)  features  are 
independent  of  r  (or  chord  length)  and  are  thus  also 
effectively  invariant  to  scale  changes  in  the  input  image. 
Similarly,  from  Kq.  (5h),  we  see  that  the  li(r)  output 
features  are  independent  of  W  (or  the  orientation  of  the 
chord  distribution)  and  are  thus  also  effectively  in¬ 
variant  to  rotational  differences  in  the  input  image. 

Thus,  as  our  feature  space,  we  use  the  (54  wedge-ring 
detector-sampled  outputs  of  the  generalized  chord  ob¬ 
servation  space.  'Phis  constitutes  a  64-sample  repre¬ 
sentation  of  the  hirjl)  chord  distribution.  Such  a 
representation  of  an  image  /(. v,y)  is  equivalent  (in 
conventional  pattern  recognition  terminology*)  to  the 
representing  of  an  image  as  a  vector  f  of  dimensionality 
l)  -  64.  with  the  elements  of  f  being  the  64  h(»)  and  h(r) 
wedge-ring  detector-sampled  values.  This  new  optical 
architecture  thus  produces  the  desired  hirjl)  chord 
distribution  in  parallel  with  a  greatly  reduced  compu¬ 
tational  load  (compared  to  a  digital  version  of  the 
equivalent  system). 

IV.  Classifier 

As  the  second  major  issue  in  this  paper,  we  consider 
the  classifier  to  be  used  with  such  a  feature  space.  In 
conventional  pattern  recognition  and  object  classifi¬ 
cation  techniques,  the  features  (the  elements  of  the 
vector  f)  are  fed  to  a  classifier  in  which  the  class  of  the 
input  object  is  determined.  This  is  achieved  by  pro¬ 
jecting  the  measured  vector  f  onto  a  linear  discriminant 
function  vector  w  and  comparing  the  scalar  output 
obtained  to  a  threshold.  In  our  general  architecture  of 
Fig.  2,  we  use  a  parallel  optical  system  to  produce  our 
observation  space  (the  autocorrelation)  of  the  input 
object),  a  wedge-ring  detector  to  provide  the  dimen¬ 
sionality  reduced  feature  space,  and  a  digital  postpro¬ 
cessor  for  classification.  This  is  a  new  quite  general  and 
very  attractive  concept  and  architecture  for  advanced 
pattern  recognition  applications.  It  employs  the  par¬ 
allelism  of  the  optical  system  for  feature  extraction  and 
the  flexibility  and  numerical  computation  features  of 
a  digital  processor  for  classification. 

We  now  discuss  the  classifier  we  used.  We  consider 
the  use  of  a  linear  discriminant  function  (LDF)  that 
maximizes  the  Fisher  ratio  to  determine  the  linear 
combination  of  output  features  to  use  for  object  classi¬ 
fication.  Our  analysis  and  formulation  follow  that  in 
Refs.  8. 15,  and  others.  Caulfield  ct  al.]r'  also  used  the 
Fisher  criteria,  but  they  used  Fourier  transform  coef¬ 
ficients  as  the  image  features  and  used  the  Fisher  LDF 
to  synthesize  a  matched  spatial  filter.  Leger  ct  a/."’17 
used  Fukunaga-Koontz  techniques"’  and  least-mean- 
square  algorithms17  to  synthesize  a  set  of  optical  filter 
functions.  Our  approach  differs  significantly  in  concept 
and  implementation,  since  we  use  the  optical  system  to 
produce  the  features  and  a  digital  postprocessor  to 
perform  the  classification.  In  approaches  such  as  those 
in  Refs.  16  and  17,  the  location  of  the  output  correlation 
peak  indicates  the  class  of  the  input  object  rather  than 
the  location  of  the  object  (as  in  a  conventional  correla¬ 
tor).  In  such  systems,  a  new  filter  function  must  be 
produced  for  each  new  class  of  object  to  be  recognized 


or  for  each  new  type  of  classifier  used  Our  system  is 
more  general  and  flexible;  since  its  outputs  can  directly 
be  used  in  any  classifier  for  any  object  class,  we  choose 
to  use  the  Fisher  criteria  because  of  their  intuitive  ap¬ 
peal. 

in  many  classifiers,  image  training  sets  (images  of 
each  class  of  object  with,  for  example,  (l  iferent  geo¬ 
metrical  distortions)  are  used  to  determine  the  LDF. 
We  consider  a  two-class  pattern  recognition  problem 
with  the  image  training  set  being  |f  |, .  j  and  If  j,  |  for  classes 
1  and  2.  respectively.  We  denote  the  number  of  training 
set  images  in  each  class  by  .V,  and  ;\ L.  respectively.  For 
each  image,  we  calculate  its  feature  vectors  f )(  and  f-  . 
We  also  calculate  the  vector  means  mi  and  m_-  for  the 
two  classes  of  training  set  data,  where  for  class  ] 

nt.  =  1 1  \  1 i fi,  iiu 

We  also  calculate  the  scatter  matrices  S|  and  S-j  for  each 
class,  where  for  class  1 

S|  =  if i.  -  m i »( f i .  -  m o'  iTi 

In  Rqs.  (6)  and  (7),  the  summation  is  over  the  i  =  .V,  or 
N->  images  in  the  corresponding  training  set. 

The  purpose  of  a  linear  classifier  is  to  determine 
(from  the  training  set  data)  an  LDF<>r  vector  w  th ■>*  i- 
a  linear  combination  of  the  observed  features.  After 
this  has  been  done,  the  system  is  tested.  In  such  a  case, 
a  linear  combination  of  the  observed  features  for  an 
unknown  input  image  f  is  calculated  and  compared  to 
a  threshold  level  T.  This  is  equivalent  to  calculating 
the  vector  inner  product  w7  f  or  f 7  w  or  the  projection 
of  f  onto  w  (all  vectors  are  column  vectors,  the  super¬ 
script  T  denotes  the  transpose  or  a  row  vector)  and 
comparison  of  w  7  f  to  the  threshold  T.  The  LDF  vec¬ 
tor  w  and  the  threshold  T  are  selected  from  the  training 
set  data  (as  we  detail  later).  If  w  7  f  g  7’.  we  classify  the 
unknown  input  object  as  a  member  of  class  one  or  two. 
respectively. 

To  determine  w.  we  must  first  select  the  performance 
measure  to  be  optimized.  In  a  Fisher  classifier,  the 
parameter  maximized  is  the  Fisher  ratio 

;  _  i'l'i  -  m,,|-  (S| 

N;  +  .S' is 

In  Kq.  (8).  m  j  and  m_>  are  the  means  of  the  projections 
of  the  training  set  data  in  classes  one  and  two  on  w.  i.e., 
for  class  1 

m  i  =  ( 1  .V 1 1  w  1  f  i ,  =  w 1  m  i .  1 9 1 

and  N  i  and  ,5'_>  are  the  si  alter  of  the  projections  for  the 
training  set  data  m  classes  one  and  two,  respectively;  i.e.. 
for  class  1 

S\  =  Pro  ject  ions  i,  -  ru ,  T 

=  lilw'T,,  -  w rm 1 1-  =  w'S,w  (till 

The  Fisher  ratio  J  in  Kq.  (8)  is  an  intuitively  pleasing 
performance  parameter.  In  a  hvperspace  data  repre¬ 
sentation.  each  input  image  is  a  vector  described  by  a 
given  position  in  this  hyperspace.  The  numerator  in 
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Kq.  (8)  is  the  difference  in  the  means  of  the  projections 
for  the  two  classes  of  training  data,  and  the  denominator 
is  the  sum  of  the  scatter  of  the  projections.  Thus  the 
numerator  in  -J  is  a  measure  of  the  separation  of  the 
classes,  and  the  denominator  is  a  measure  of  the  clus¬ 
tering  within  each  class.  The  reader  should  recall  that 
the  parameters  in  Kqs.  (6)-(  10)  refer  to  the  training  set 
data  for  the  two  classes. 

To  determine  the  LDF  w  that  maximizes  J  in  Kq.  (8), 
we  must  describe  J  in  terms  of  w.  Using  Kqs.  (6)  and 
(9).  we  can  write  the  numerator  in  Kq.  (8)  as 

(nil  -  m-j I-  =  Iw  'mi  -  w 7  m_.l-  =  w 7Sj, w.  (Ill 

where  the  between-dass  scatter  matrix  S/j  for  the 
two-class  discrimination  problem  is  the  vector  outer 
product: 

Ah  =  Imi  -  irijMmi  -  m_>)7  lit!) 

We  describe  the  denominator  in  Kq.  (8)  using  Kqs.  (7) 
and  ( 10)  for  the  two-class  problem  as 

.S’ ;  +  .S’H  =  W  7  S  ]  W  +  W  7  SjW  —  W  7  S a  W .  ( 1  :l  I 

where  the  within-class  scatter  matrix  Svv-  is 

Sjc  =  S|  +  Sj.  (141 

Substituting  Kqs.  (11 )  and  ( 18)  into  Kq.  (8),  we  find  that 
the  Fisher  ratio  can  be  described  in  terms  of  the  LDF 
w  as 

./( w )  =  w  rS/i  w/h  rS«-w.  (I'O 

Differentiating  Kq.  (15)  with  respect  to  at  and  setting 
the  result  equal  to  zero,  we  find  that  w  is  the  solution 
of  the  general  eigenvalue  problem 

Sow  =  XSwW  (Hint 

If  Sir  is  nonsingular,  we  can  write  Kq.  (16a)  as 

S^'S/jW  =  Aw.  ( liilil 

The  unique  solution  to  Kq.  (16b)  is  our  LDF.  It  is  given 
by 

w  =  S iv1 1 m  1  -  m  A.  ( ITl 

Thus  to  determine  the  LDF  in  Kq.  (17)  we  must  calcu¬ 
late  the  vector  means  mi  and  mj  plus  the  within-class 
scatter  matrix  Sn  for  our  training  set  data.  Substi¬ 
tuting  these  into  Kq.  (17)  then  yields  the  optimal  LDF 
w  that  maximizes  the  Fisher  ratio -./(w)  in  Kq.  (8).  As 
our  threshold,  we  use  the  average  of  the  means  of  the 
projections  for  our  training  set  data,  i.e.. 

T  -  (no  -r  nijl/'J.  ( 1X1 

V.  Initial  Experimental  Verification 

A.  Available  Data  Base 

To  demonstrate  the  use  of  this  Fisher-discriminant 
classifier  with  a  wedge-ring  detector-sampled  general¬ 
ized  chord  transformation  observation  space,  we  used 
an  available  IR  ship  image  data  base.  This  data  base 
contained  two  classes  of  ships  with  86  images  in  each 
class  (one  image  e''ery  10°  with  a  90°  depression  angle). 


(iray -scale  and  binary  images  were  used  in  all  experi¬ 
ments,  and  the  hull  of  the  ship  was  removed  (retaining 
only  the  superstructure).  The  number  of  features 
(wedge-ring  detector  samples)  or  the  dimensionality  I) 
of  the  feature  space  used  affects  the  number  of  training 
set  images  iV|  +  JVj  =  N  that  are  necessary.  The  cases 
to  consider  are  I)  >  N.  I)  =  N,  and  I>  <  N.  If  I)  >  N. 
many  solutions  are  possible,  and  we  simply  pick  one.  In 
I)  >  A7,  we  find  that  Sk  is  singular  (its  rank  is  less  than 
/>),  and  thus  an  LDF  w  to  maximize  the  Fisher  ratio 
does  not  exist  (since  Sjv  does  not  exist),  and  instead 
many  solutions  exist.  If  I)  =  N,  a  unique  solution  exists 
and  is  easily  obtained  by  solving  a  matrix- vector 
problem.  Il'/f  <  N ,  no  direct  solution  exists,  and  thus 
we  approximate  one  by  least-mean-square  techniques. 
Since  maximizing  the  Fisher  ratio  is  knmvn*  to  be 
equivalent  to  a  least-mean-square  solution,  use  of  such 
an  approach  automatically  requires  N  >  I).  We  thus 
find  the  soution  w  that  minimizes  the  least-square 
error. 

B.  Dimensionality  Reduction 

The  consequence  of  the  N  >  l)  requirement  (which 
is  the  practical  case  of  concern  in  pattern  recognition) 
is  that  if  our  available  image  data  base  is  of  finite  size 
(this  is  obviously  the  practical  situation),  we  must  thus 
reduce  I)  before  a  Fisher  or  similar  classifier  can  be 
designed  and  before  significant  test  data  are  possible. 
To  quantify  these  remarks,  we  consider  our  present 
problem  and  available  data.  If  we  retain  all  l)  =  64 
wedge-ring  detector  features,  we  require  N  >  65 
training  set  images.  For  a  two-class  problem,  we  only 
have  available  86  images  per  class  or  72  images  total. 
Thus,  if  all  64  wedge-ring  detector  values  are  retained 
as  features,  we  would  have  to  use  65  of  our  available  72 
images  for  training.  This  would  leave  us  with  only  7  of 
the  72  images  for  use  as  new  images  during  testing.  The 
results  of  such  experiments  (training  on  65  of  72  images) 
would  not  be  impressive.  Thus  we  next  consider  fur¬ 
ther  dimensionality  reduction  of  our  feature  space. 

To  perform  more  impressive  tests  and  to  reduce  the 
size  of  the  training  set  and  the  amount  of  calculations 
required,  the  number  of  features  used  must  be  reduced. 
Such  dimensionality  reduction  is  commonly  employed 
(for  different  reasons,  i.e..  to  reduce  the  computations 
required).  From  an  analysis  of  the  wedge-ring  detector 
outputs  for  our  full  set  of  72  images,  we  found  that  the 
outer  8  ring  readings  were  approximately  zero.  We  thus 
ignored  these  features  in  all  cases  and  reduced  the 
maximum  number  of  features  to  /)  =  56.  For  a  feature 
vector  of  dimension  56.  we  would  still  require  29  images 
in  each  class  for  training.  To  further  reduce  the  di¬ 
mensionality  of  our  feature  space,  we  calculated  the 
means  and  m and  the  variances  rrid  and  <r],/  of 

each  wedge  and  ring  detector  output  (d  =  1 . 56)  for 

various  two-class  training  sets  of  data.  We  then  cal¬ 
culated  R  =  (ni|rf  —  m  >,i)/(<T fri  +  (T4ri)  for  each  case  for 
all  56  detectors.  As  our  features,  we  retained  the  18 
detector  outputs  with  the  largest  mean  difference  to 
variance  ratio  R  for  the  various  training  sets.  Our 
simplified  technique  for  reduction  of  the  dimensionality 
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Fig.  «}.  Representative  images  for  the  class  1  ship  imagery:  (top 
right)  0.  (top  left)  45.  (bottom)  90°. 
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Eig.  4.  Representative  images  for  the  class  2  ship  imagery:  (top 
right)  I),  (top  left)  45,  (bottom  90°. 


of  the  feature  space  assumes  independent  features.  A 
more  precise  technique18  using  the  covariance  of  the 
features  could  be  employed  (to  account  for  dependent 
relationships  between  features).  However,  our  results 
were  adequate,  and  thus  such  techniques  were  not  em¬ 
ployed. 

c.  Image  Training  Sets 

In  Figs.  It  and  4.  we  show  three  of  the  images  from  the 
set  of  36  images  for  each  of  the  two  classes  of  ships  used. 
We  number  the  156  images  in  each  class  consecutively 
from  the  how  to  the  stern,  with  the  class  one  images 
denoted  by  1-36  and  the  class  two  images  by  37-72. 
Images  1  and  37  are  the  bow  views,  images  9, 27, 46,  and 
64  are  the  broadside  views,  etc.  We  selected  five 
training  sets  of  data  (Table  I).  The  same  aspect  views 
were  used  for  each  ship  class  for  compatibility,  and  thus 
only  the  ship  image  numbers  for  class  1  are  listed  in 
Table  I.  Training  set  1  contains  all  the  imagery. 
Training  set  2  contains  half  of  the  imagery  (images  every 
20° ).  Three  sets  (sets  3-5)  of  1 2  training  set  images  per 
class  were  chosen  In  set  3,  the  training  images  were 
clustered  within  ±50°  of  broadside  on  the  front  and 
back  of  the  ship.  In  set  4,  the  1 2  images  were  clustered 
every  10°  from  40°  to  150°  on  one  side  of  the  ship  only, 
and  in  set  5  they  were  evenly  spaced  every  30°. 


Table  I.  Image  Training  Sets  Used 


Image  Number  of 
training  training  set 
set  images  used 
number  per  class 

Training  set 
images  used 
for  class  1 
(class  2 
analogous) 

Training  set 
remarks 

) 

36 

1,2 . 35,36 

AJJ  images 

2 

18 

1,3,5 . 35 

Images  every  20' 

:i 

12 

4,6,8 . 12,14:22,24, 

.  .  .  ,30.32 

Images  every  20' 
(within  ±50° 
of  broadside) 

4 

12 

4,5 . 14.15 

Images  every  l()c 
(one  side  only, 
near 

broadside) 

5 

12 

1,4 . 13,16;  19.22. 

...,31.34 

Images  every  .X()‘ 

Table  II. 

Simulated  Test  Data  (Binary  Imagery) 

Experiment 

number 

Fisher 

ratio 

Number 
of  errors 

Probability 
of  correct 
classification 
(<*) 

i 

114 

0 

UK) 

2 

126 

0 

too 

3 

1180 

8 

88.9 

4 

2322 

4 

94.4 

5 

158 

0 

1(H) 

Simulation  Results 

In  Table  II,  we  list  our  experimental  results  obtained 
for  binary  ship  image  data  with  the  hull  of  the  ship  re¬ 
moved.  (This  was  done  to  remove  nonreliable  gray¬ 
scale  image  data,  and  hence  the  binary  version  of  such 
imagery  was  used  in  our  present  tests  also.)  In  all  cases, 
the  18  wedge-ring  detector  features  previously  noted 
(Sec.  V.B)  were  used.  The  five  experiment  numbers 
listed  correspond  to  the  tests  performed  using  the  cor¬ 
responding  five  image  training  sets  noted  in  Table  I.  In 
each  case  we  calculated  mi,  m2,  and  S(y  for  the  indicated 
training  set  data  and  from  these  computed  the  LDF  w 
in  Eq.  (17)  and  the  threshold  T  in  Eq.  (18). 

From  experiment  1.  we  note  that  the  entire  data  base 
was  used  for  training  and  that  no  errors  occurred.  This 
indicates  that  the  full  data  base  is  linearly  separable 
using  as  few  as  18  features.  It  is  thus  realistic  to  assume 
that  some  subset  (less  than  all  36  images  per  class)  of  the 
full  image  data  base  might  provide  similar  performance 
to  that  obtained  using  the  full  data  base  (i.e..  1  ()0'7 
probability  of  correct  recognition).  One  cannot  theo¬ 
retically  prove  this,  however,  but  test  data  seem  to  verify 
this  intuitive  remark.  In  experiment  2,  we  use  18  im¬ 
ages  per  image  class  and  find  that  the  computed  LDF 
is  again  capable  of  correctly  classifying  all  72  images  in 
both  classes  (half  of  which  it  had  not  previously  seen) 
with  no  errors. 

Experiments  3-5  consider  the  use  of  a  much  smaller 
training  set  (12  images  per  class  or  one-third  of  the 
available  imagery)  and  the  testing  of  the  computed  LDF 
against  all  72  images.  In  experiment  3, 12  images  per 
object  class  were  used  (clustered  around  t  he  broadside 
views  of  the  target,  from  both  sides  of  the  ship).  Good 
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results,  8  errors  out  of  72  images  or  88.9°;  correct  rec¬ 
ognition,  were  obtained.  As  might  be  expected,  most 
of  the  8  errors  occurred  for  test  images  close  to  the  bow 
and  stern  view's.  Since  such  images  have  fewer  pixels 
on  the  target  and  since  these  images  were  not  included 
in  the  test  data,  these  errors  are  to  be  expected.  In  a 
ship  recognition  scenario  with  such  imagery,  one  would 
not  expect  to  perform  classification  on  such  bow  and 
stern  views  of  a  ship. 

However,  to  pursue  further  the  issue  of  training  set 
selection  and  the  performance  of  this  feature-extraction 
space,  we  considered  other  training  sets  of  1 2  images  per 
class  (from  our  data  base  of  86  images  per  class).  In 
experiment  4, 12  images  taken  from  only  one  side  of  the 
ship  were  used  during  training.  The  motivation  for  this 
choice  was  that  the  views  from  both  sides  of  the  ship 
were  rather  similar,  and  thus  we  could  possibly  contain 
all  information  on  the  ship  class  by  training  on  images 
from  one  side  of  the  ship  only.  Our  results  (4  rather 
than  8  errors  out  of  72  images,  48  of  which  were  never 
seen  in  the  training  of  the  classifier)  verified  this  con¬ 
jecture.  As  might  have  been  expected,  the  majority  of 
errors  encounteed  again  occurred  for  images  near  the 
bow  and  stern.  These  images  were  not  present  in  the 
training  set.  and  thus  such  errors  are  again  expected. 
As  our  third  set  of  twelve  training  set  images  per  class, 
we  used  set  5(12  images,  evenly  spaced  every  80°  about 
the  full  ship).  Our  performance  (no  errors  for  all  72  test 
set  images)  is  the  hest  possible  (matching  that  obtained 
in  experiments  1  and  2  using  86  and  18  training  set  im¬ 
ages,  respectively,  per  class). 

In  all  cases,  no  more  than  8  errors  occurred  (above 
88.9T  probability  of  correct  recognition).  The  third 
major  issue  of  concern  in  this  paper  is  further  analysis 
of  these  results  to  obtain  insight  into  why  such  excellent 
performance  results  and  insight  into  how  selection  of 
the  training  set  used  affects  the  results  obtained. 

Before  leaving  our  experimental  data  section,  we 
present  (Table  III)  our  ship  data  test  results  obtained 
using  gray-scale  imagery  (for  the  same  experiments  and 
training  sets  used  for  binary  imagery).  These  results 
are  quite  similar  to  those  obtained  using  binary  imagery 
(Table  II)  with  a  reversal  in  the  results  for  tests  8  and 
4  being  the  most  obvious  differences.  (We  attribute 
this  to  additional  reliable  gray-scale  image  data  present 
in  the  superstructure  and  deck  line  of  the  ship.)  These 
experimental  results  are  noteworthy  because  they 
represent  the  generalized  chord  transform  operation 
that  we  noted  (Sec.  II)  that  our  optical  system  could 
directly  and  easily  perform. 

VI.  Discussion 

Our  optical  processor  to  realize  a  chord-distribution 
sampled  feature  space  has  proven  most  attractive  as  has 
our  wedge-ring  detector-sampled  chord  feature  space. 
Our  technique  to  reduce  further  the  dimensionality  of 
our  feature  space  has  likewise  been  shown  to  be  useful 
and  adequate.  Further  research  is  needed  to  optimize 
our  image  training  set  selection  technique.  Our  three 
training  sets  of  data  (for  our  12  image  per  class  t  raining 
sets)  were  chosen  to  investigate  and  demonstrate  certain 


Table  HI.  Simulated  Test  Data  (Gray-Scale  Imagery ) 


Kx  per  intent 
number 

Number 
{ it errors 

Probability 
of  correct 
classification 
t'c  ) 

Training  set 
remarks 

i 

0 

10(1 

All  images 

2 

1 

9HJ) 

Images  every  JO0 

Images  every  -0° 
(within  ±')0° 
t»f  broadside) 

■t 

6 

fti.n 

Images  every  10° 
(one  side  only, 
near  broadside) 

5 

0 

100 

Images  every  H0° 

features  useful  for  training  set  selection.  We  will  am¬ 
plify  on  our  general  remarks  to  follow  in  a  future  pub¬ 
lication  when  we  consider  the  effects  of  noise  on  the 
performance  of  our  system. 

For  our  present  discussion,  we  will  consider  how  the 
Fisher  ratio  value  J  (obtained  from  training  set  data) 
can  be  used  to  influence  our  image  training  set  selection 
criteria. 

We  first  note  that  maximization  of  J  is  a  least -square 
problem,  and  thus  the  J  parameter  (Table  11)  provides 
a  measure  of  the  accuracy  of  the  least -squares  fit  solu¬ 
tion.  Specifically.  J  is  inversely  proportional  to  the 
least-squares  error.  Thus  a  low  J  value  (Table  II )  in¬ 
dicates  a  large  least-squares  error  in  our  solution. 
However  the  major  point  we  advance  at  this  junction  is 
that  a  large  J  (and  hence  a  small  least -mean-square 
error)  does  not  necessarily  reflect  that  better  classifi¬ 
cation  performance  will  result.  Rather,  a  smaller  J  for 
the  training  set  indicates  that  the  training  set  is  more 
divergent  (large  mean  square  error)  or.  equivalently, 
that  it  is  more  representative  of  the  lull  image  data 
base. 

Thus  the  philosophy  we  presently  discuss  advocates 
selection  of  a  training  set  with  a  lower  J  value.  This 
premise  is  based  upon  the  fact  (confirmed  by  our  initial 
experiments)  that  such  an  image  training  set  provides 
a  better  statistical  representation  of  the  test  data.  As 
we  noted  earlier,  such  measures  for  training  set  selection 
are  needed,  both  to  provide  more  meaningful  test  data 
and  to  provide  better  discrimination  performance. 

To  quantify  these  remarks,  let  us  consider  the  Fisher 
ratio  J  (column  2  of  Table  II)  calculated  for  the  various 
training  sets.  Considering  the  first  three  entries,  we 
note  that  J  increases  as  the  number  of  training  set  im¬ 
ages  used  increases  (114  for  all  72  images.  126  for  86 
images,  and  1 180  for  24  images).  This  reflects  the  ob¬ 
vious  fact  that  it  is  easier  to  separate  24  or  86  images 
than  72.  The  large  increase  (126-1 180)  in  going  from 
86  to  24  images  is  also  due  to  t  he  12  images  chosen  per 
class  (all  were  within  ±50°  of  broadside  and  thus  cluster 
well).  Similar  remarks  apply  to  the  large  J  (2822)  ob¬ 
tained  for  the  case  when  the  12  training  set  images  per 
class  were  chosen  from  the  same  side  of  the  ship. 

Recall  that  a  large  J  merely  reflects  a  small  least - 
squares  error  that  indicates  that  the  training  sel  sepa¬ 
rates  well  between  classes  and  clusters  well  within  each 
class.  However,  a  large  ■)  value  does  not  imply  good 
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performance  on  test  images.  In  fact,  the  opposite  effect 
occurs.  (An  LDF  formed  from  a  training  set  with  a 
large  J  yields  worse  performance  than  does  one  with  a 
low  J.)  This  is  obvious  and  provides  insight  into  a  more 
formal  rather  than  an  ad  hoc  technique  for  training  set 
selection  that  we  now  advance. 

Once  a  training  set  is  selected,  our  algorithm  produces 
an  LDF  with  the  largest  J  possible  (for  the  given 
training  set).  The  value  of  J  obtained  indicates  the 
divergence  of  the  training  set.  Thus  an  LDF  with  a  low 
■J  indicates  that  the  training  set  used  is  more  statistically 
representative  (larger  divergence)  of  the  imagery  than 
is  an  l.DF  with  a  larger  J.  Once  we  have  chosen  the 
number  of  training  set  images  we  desire  to  use  (e.g..  12 
per  class),  we  can  use  J  as  a  measure  of  which  training 
set  and  which  LDF  is  best.  Specifically,  we  choose  the 
training  set  and  LDF  with  the  lowest  J. 

Comparing  J  for  the  three  training  sets  of  12  images 
per  class  (experiments  3-5),  we  find  a  low  J  ( 158)  when 
the  12  training  images  per  class  were  evenly  distributed 
(every  30°)  around  the  entire  ship  rather  than  clustered 
on  one  side  of  the  ship  (experiment  4,  J  =  2322)  or  near 
broadside  on  both  sides  of  the  ship  (experiment  3,  J  = 
1180).  Intuition  also  tells  us  that  training  set  5  should 
be  more  statistically  representative  of  the  image  classes. 
(The  -J  value  reflects  and  quantifies  this.)  Our  exper¬ 
imental  data  confirm  this.  [The  LDF  produced  from 
training  set  5  yields  fewer  errors  (zero)  than  did  the 
LDFs  in  cases  3  and  4.|  This  technique  for  training  set 
selection  is  directly  applicable  to  other  feature  spaces 
and  other  classifiers.  We  will  elaborate  further  on  this 
issue  and  other  methods  of  training  set  selection  that 
explain  the  classification  percentages  in  experiments 
3  and  4  in  a  future  publication. 

Care  must  still  be  exercised  in  selecting  the  number 
of  training  set  images  N  when  D  features  are  used. 
Specifically,  to  use  a  least-mean-square  solution,  we 
require  N  >  l).  However,  the  N  training  set  images 
must  be  totally  independent.  This  is  difficult  to  achieve 
in  practice,  and  hence  we  do  not  choose  N  =  D  +  1  but 
rather  utilize  several  additional  training  set  images. 
When  l)  was  large  (e.g.,  56  features),  we  found  the  sum 
of  the  covariance  matrices  for  both  classes  to  be  very 
unstable  with  digital  truncation  error  alone  often  being 
sufficient  to  affect  the  rank  of  the  matrix  and  permit  its 
inversion.  In  general,  operating  with  a  smaller  di¬ 
mensionality  feature  space  is  also  preferable  for  com¬ 
putational  reasons,  but  this  is  not  the  major  issue  in  an 
optical  processor. 

We  have  considered  other  simpler  observation  spaces 
such  as  the  magnitude  of  the  Fourier  transform  of  the 
input  object.  The  wedge-ring  detector  was  originally 
intended  to  be  used  in  Fourier  plane  sampling.  Since 
the  information  content  of  the  power  spectrum  and  the 
autocorrelation  are  similar  (the  latter  is  the  Fourier 
transform  of  the  former),  it  might  appear  that  there  is 
no  difference  in  which  plane  we  sample.  However,  the 
physical  significance  of  the  two  representations 
(wedge-ring  detector  samples  of  the  Fourier  transform 
and  the  autocorrelation)  is  quite  different.  We  make 
no  effort  to  decide  which  is  best  for  pattern  recognition. 


We  note  that  the  sampling  of  the  autocorrelation  is 
equivalent  to  the  chord  histogram.  We  also  note  that, 
by  blocking  dc  in  forming  the  autocorrelation,  the 
conventional  chord  distribution  is  produced  (corre¬ 
sponding  to  the  boundary  of  the  object).  More  light 
level  and  a  less  narrow  autocorrelation  (with  more 
structure)  result  from  our  generalized  chord  transform. 
Hence  this  appears  preferable  and  to  be  a  more  attrac¬ 
tive  observation  space  for  an  optical  processor. 

Our  architecture  (in  which  the  features  of  the  input 
object  are  optically  computed)  is  quite  general  and  at¬ 
tractive.  It  utilizes  the  high-speed  and  parallel-pro¬ 
cessing  features  of  an  optical  system  for  feature  ex¬ 
traction,  and  it  allows  these  optically  computed  features 
to  be  fed  to  any  output  classifier.  Discussions  of  other 
optical  pattern  recognition  systems  using  the  Fisher, 
Fukunaga-Koontz,  least -mean-square,  and  other  clas¬ 
sifiers  were  advanced  in  Sec.  IV.  However,  these  sys¬ 
tems17’  17  used  such  classifiers  to  synthesize  the  filter 
used.  Thus  they  all  require  different  filters  to  be  made 
for  every  different  pattern  recognition  application  and 
new  filters  to  be  made  if  different  classifiers  are  used  (as 
is  required  in  certain  applications).  Our  architecture 
is  more  general,  since  the  optical  system  need  not 
change  if  different  pattern  recognition  problems  are 
being  addressed  or  if  different  classifiers  are  to  be 
used. 

Another  important  aspect  of  our  approach  can  he 
seen  by  considering  the  application  of  the  Fisher  criteria 
to  synthesis  of  a  matched  spatial  filter  using  a  Fourier 
coefficient  feature  space.  In  such  an  approach,1’’  the 
LDF  w  is  a  linear  sum  of  the  Fourier  coefficients  of  the 
training  set  data,  and  thus  a  matched  spatial  filter  of 
this  2-D  LDF  can  be  fabricated  and  placed  at  the  Fou¬ 
rier  transform  plane  of  a  frequency  plane  correlator. 
The  output  correlation  plane  in  this  case  would  consist 
of  peaks  of  light  of  amplitude  proportional  to  m  i  or  m> 
(the  means  of  the  projections  for  the  data  in  classes  1 
and  2,  respectively).  If  the  data  in  the  input  were  in 
class  1.  the  output  would  have  an  amplitude  propor¬ 
tional  to  nii.  If  the  data  were  of  class  2.  the  output 
would  be  proportional  to  m ■>  and.  with  the  location  of 
each  peak  of  light,  proportional  to  the  location  of  each 
object  in  the  input  field  of  view.  However,  it  has  been 
shown  in  Ref.  8  that  maximizing  the  Fisher  ratio  forces 
the  output  projection  values  for  the  two  classes  of  data 
to  be  ±1.  respectively.  Although  the  least -squares  so 
lut  ion  will  not  give  projections  of  exactly  ±  1.  it  will  (with 
high  probability)  give  positive  and  negative  means  for 
the  two  classes.  Thus  the  intensity  output  from  such 
an  optical  correlator  will  not  allow  one  to  distinguish  the 
sign  or  polarity  of  the  output  projections  and  hence  will 
not  permit  target  classification.  Heterodyne  detection 
of  the  correlation  plane  output  is  thus  necessary  if  a 
Fisher  criterion  is  used  for  synthesis  of  the  matched 
spatial  filter  in  a  correlator.  Only  with  such  methods 
can  the  shift-invariant  advantages  of  such  systems  (over 
our  feature-extraction  method)  be  realized.  In  our  use 
of  the  Fisher  criteria  (in  which  the  classification  is 
performed  in  a  digital  postprocessor),  different  pro¬ 
jection  polarities  are  easily  handled  and  are  in  fact 
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preferable.  Our  algorithm  and  general  architecture  can 
also  be  directly  extended  to  multiclass  pattern  recog¬ 
nition  applications  (involving  more  than  two  classes  of 
objects)  more  easily  than  can  other  optical  architectures 
(that  require  a  different  optical  system  to  be  used  when 
different  numbers  of  classes  of  data  are  involved). 

VII.  Summary 

In  this  paper,  we  have  described  a  new  optical  system 
that  computes  the  histogram  of  the  generalized  chord 
transformation  of  the  input  object.  Our  wedge-ring 
detector-sampled  feature  space  provides  a  reduced  di¬ 
mensionality  vector  representation  of  the  input  object 
that  has  the  fundamental  data  necessary  for  scale  and 
rotational  invariance.  As  our  classifier,  we  chose  the 
linear  discriminant  funct  ion  that  maximizes  the  Fisher 
ratio.  Our  initial  experiments  showed  that  we  could 
recognize  and  correctly  classify  the  input  image  data 
with  as  few  as  18  features  and  that  using  the  proper 
training  set  of  12  images  per  class,  we  could  achieve 
100%  correct  recognition  of  the  72  images  available  in 
our  data  base.  Extensions  of  this  approach  to  multi¬ 
class  pattern  recognition  were  noted  together  with  the 
use  of  other  output  classifiers  and  the  initial  ideas  for 
techniques  to  select  the  optimal  training  set  of  data  for 
any  such  system. 
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Efficient  approach  to  designing  linear  combination  filters 


B.  V.  K.  Vijaya  Kumar 


\  arious  linear  cum  liinat  ions  of  simple  matched  spatial  filters  have  been  proposed  in  the  literature  to  im¬ 
prove  the  discrimination  in  multielass  pattern  recognition.  It  has  been  shown  that  all  such  approaches 
based  on  deterministic  constraints  can  be  modeled  as  similar  matrix/veotor  problems,  the  only  differences 
arising  in  the  individual  constraint  vectors.  Since  the  design  of  any  of  these  linear  combination  filters  <t,(T'l 
can  he  posed  as  a  common  matrix/vector  problem,  efficient  iterative  methods  can  he  used  to  determine  the 
l.l'Ks.  The  application  of  one  such  method  called  the  modified  hyperplane  (MHD  method  for  determining 
the  1,('K  is  described  and  its  convergence  behavior  is  numerically  investigated  for  a  set  of  seven  patterns.  It 
is  shown  that  the  MHl’  method  yields  correct  l/T's  (with  rms  error  <o.l'< )  in  less  than  ten  iterations. 


I.  Introduction 

Matched  spatial  filters  (MSF)  have  been  extensively 
used  in  optics1  to  identify  a  known  pattern  in  a  noisy 
background.  This  simple  approach  has  been  shown  to 
fail  in  the  presence  of  common  image  distortions  such 
as  in-plane  rotations  and  sca'e  changes.  Simple  MSFs 
are  also  seen  to  he  inadequate-  for  multiclass  pattern 
recognition  applications.  Multiclass  pattern  recogni¬ 
tion  fMPR)  can  be  defined  as  t  he  task  of  discriminat  ing 
a  pattern  from  among  many  classes  of  patterns.  A  good 
example  of  MPR  is  the  identification  of  a  given  letter 
in  a  language  from  its  alphabet. 

Linear  combinations  of  MSFs  matched  to  the  indi¬ 
vidual  patterns  have  been  used  to  improve  the  dis¬ 
crimination  in  MPR.-  Such  an  approach  resulted  not 
only  in  improved  discrimina  'on  between  patterns  hut 
also  in  increased  tolerance  to  geometrical  distortions  in 
a  given  pattern  class.  Hester  and  Casasent1  have  shown 
through  the  use  of  the  Gram-Schmidt  procedure  that 
a  singl"  spatial  filter  can  be  designed  to  produce  iden¬ 
tical  cross-correlation  values  with  all  the  patterns  in  the 
input  set.  For  obvious  reasons,  this  type  of  filter  is 
known  as  an  equal  correlation  peak  (KCP)  filter. 
Braunecker  ct  at.  ‘  utilize  multiple  linear  combination 
filters  to  produce  compact  binary  descriptions  of  the 
input  pattern.  All  these  filters  are  purely  deterministic 
in  the  sense  that  they  depend  only  on  the  training  set 
patterns.  Principles  of  statistical  pattern  recognition 
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were  used  by  Caulfield  and  Haimes-'  to  produce  statis¬ 
tically  optimal  LCFs.  The  common  nature  of  all  these 
LCFs  is  well  unified  by  Caulfield.1’ 

In  this  paper  we  utilize  the  unifying  notation  of 
Caulfield1'  and  our  earlier  work7  which  showed  that 
these  previous  LCF  approaches  can  he  reduced  to  a 
common  matrix/vector  problem.  We  note  that  the  size 
of  this  matrix/vector  problem  is  directly  proportional 
to  the  number  of  patterns  in  the  input  training  set  and 
that,  since  the  accuracy  and  the  usefulness  of  a  LCF 
increases  with  the  size  of  the  input  training  set,  we  need 
to  solve  larger  matrix/vector  problems  for  better  accu¬ 
racy.  The  contribution  of  this  paper  is  that  this  refor¬ 
mulation  of  the  LCF  design  into  a  single  matrix/vector 
solution  can  benefit  from  the  vast  amount  of  results 
available8  for  solving  matrix/vector  problems.  As  an 
illustration  of  this  approach,  we  propose  the  use  of  t  be 
modified  hyperplane  (MHP)  method  to  design  the 
LCFs.  The  MHP  was  initially  used  by  Ramakrishnam 
ct  at.'1  for  image  restoration  and  by  Vijaya  Kumar1"  for 
adaptive  array  null  synthesis.  In  1  his  paper  we  point 
out  the  usefulness  of  this  technique  in  a  LCF  synthesis 
for  MPR. 

II.  Linear  Combination  Filters 

Consider  the  set  of  N  input  characters 
tXi.Xj, ....  X.\|.  Kach  pattern  X,  is  converted  from 
an  image  to  a  vector  for  convenience.  The  length  of  the 
vectors  equals  the  total  number  of  pixels  in  the  patterns. 
A  linear  combination  filter  Y  is  defined  as 

Y’  =  n  i  X  i  +  «  -X  ■  -f  . . .  +  a  \  X  \ .  ()) 

where  . a \  are  real  values  denoting  the  con¬ 

tribution  of  each  input  pattern  in  the  LCF  Y.  The  LCF 
designs  discussed  below  describe  three  techniques  Idr 

determining  the  coefficients  rti.n. . a\.  These 

techniques  differ  only  in  the  constraints  imposed  for 
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determining  these  weights  and  the  number  of  filters 
used.  'I'he  N  mutually  orthogonal  filters  (MOP) 
Y|,Y^ . Y.v  are  designed  such  that  each  filter  cor¬ 

relates  with  only  one  pattern  and  produces  zero  cross 
correlation  with  all  other  inputs,  i.e., 

Y,  •  X,  =  ()„.  c„>) 

where  the  dot  denotes  the  inner  product  of  the  two 
vectors  and  fip  denotes  the  Dirac  delta  function.- 

'I'he  ECP  filter  approach  suggested  by  Hester  and 
C’asasent1'  requires  that  a  single  LCF  Y  he  determined 
that  correlates  equally  well  with  all  the  patterns  in  the 
training  set,  i.e., 

Y  •  X,  =  I  for  /  =  1,2 . V.  c<) 

In  the  nonredundant  filter  (NRP)  method1  M  LCFs 

Y),Yg . Y\/  are  chosen  such  that,  when  all  these  M 

filters  are  correlated  with  a  given  input  X,,  the  resulting 
M  cross-correlation  values  provide  an  Af-tuple  de¬ 
scription  of  /,  the  input  number.  If  M  is  greater  t han 
logjA/,  we  can  see  that  the  resulting  description  is  a  bi¬ 
nary  one.  To  illustrate  this  method,  consider  an  input 
set  of  three  patterns  |X|.X-j,X;,|  and  two  nonredundant 
filters  Y  i  and  Yj.  Since  the  binary  descript  ions  of  the 
numbers  1,2,3  are  (0,1), (1,0), (1,11,  respectively,  Y,  and 
Yj  should  satisfy 

Y,  •  X,  =0;  Y,  •  X,  =  1: 

Y,»X,=  I:  Yj»X.  =  0;  cu 

Y,  •  X:,  =  I:  Y.  •  X,  =  I . 

The  LCF  solutions  to  Eqs.  (2)-(4)  have  been  shown 
to  be  described7  in  terms  of  the  solution  a  to 

Ka,  =  u,„.  (‘>1 

For  the  MOP,  u,„  =  up  is  an  N -element  vector  with  a 
1  in  the  i  th  position  and  zeros  everywhere  else.  R  is  the 
N  X  N  correlation  matrix  of  the  input  data 

IX, . X.\  |.  For  the  KCP,  u,„  =  u  is  an  N-element 

vector  containing  all  ones.  For  the  nonredundant  fil¬ 
ters,  u,„  =  u-j,  is  a  vector  consisting  of  ones  and  zeros 
only,  corresponding  to  the  / 1 h  bits  in  the  binary  de¬ 
scriptions  of  numbers  1,2 . N.  In  our  example  N 

=  3,  M  =  2,  and  u-j,  would  be  given  as  [0  1  1| 1 ,  where  the 
superscript  T  denotes  transpose.  Similarly  Uj_>  is  given 
as  1 1  0  1 1  r.  A  nonbinary  descript  ion  such  as  ternary  can 
also  be  used  to  reduce  the  number  M  of  the  filters  but 
at  the  expense  of  increased  dynamic  range  requirements 
in  the  output  plane  of  the  correlators. 

From  this  we  see  that  all  three  techniques  reduce  to 
the  same  matrix/vector  problem  in  Kq.  (5).  Only  the 
constraint  vectors  u,,,  u,  and  u_>,  differ  in  the  three 
cases.  'Phis  leads  to  the  possibility  of  solving  all  three 
problems  with  only  a  few  changes.  For  good  filters  we 
require  a  large  input  data  set,  thus  increasing  the  value 
of  N.  Since  the  size  of  the  matrix  R  is  N  X  N.  we  need 
to  consider  efficient  methods  of  solving  for  a. 

III.  Modified  Hyperplane  (MHP)  Method 

'I'he  matrix/vector  Kq.  (5)  can  be  solved  by  finding 
the  inverse  of  the  matrix  R.  However.  N  is  often  so 


large  that  determining  the  inverse  of  R  directly  can  be 
computationally  cumbersome.  Projection  techniques1 1 
are  usually  suggested  for  solving  such  equations  when 
N  is  large.  In  projection  methods,  the  N  constraints  in 
Kq.  (5)  in  terms  of  the  unknown  coefficients 
o,,aj, ...  ,  «v  are  considered  to  represent  .V  hyper¬ 
planes  in  an  jV-dimensional  hyperspace.  The  .V  coor¬ 
dinates  of  the  intersection  of  these  iV  hyperplanes  are 
the  N -coefficient  values. 

'I'he  alternating  orthogonal  projection  (AOI’I  method 
suggested  by  Youla1 1  projects  the  initial  guess  (solution) 
vector  orthogonally  on  all  N  hyperplanes,  one  after 
another.  Ramakrishnam  ct  al.'-'  suggested  a  modifi¬ 
cation  to  this  method  to  improve  its  convergence  be¬ 
havior.  This  modification  involves  rearranging  the  .Y 
hvperplanes  of  our  equations  so  that  the  adjacent  hy¬ 
perplanes  are  orthogonal  to  each  other.  This  modifi¬ 
cation  was  shown1"  to  be  preferable  to  AGP  both  in 
terms  of  the  computational  load  per  iteration  and  in 
terms  of  the  number  of  iterations  needed  for  conver¬ 
gence. 

Let  us  consider  Kq.  (5)  with  u,„  =  u  (a  unit  vector)  for 
illustration  purposes.  'I'he  other  two  equations  can  be 
handled  in  a  similar  manner.  Let  us  denote  the  .V  rows 

of  the  matrix  R  by  the  vectors  Si  .S-_> . S\.  Then  the 

N  hyperplanes  in  the  At -dimensional  space  are  repre¬ 
sented  by 

S/  a  =  «(/  I.  nil 

where  «(/)  is  the /th  element  in  the  constraint  vector  u. 
The  two  steps  in  the  MHP  are  detailed  in  the  next  two 
subsections. 

A.  Hyperplane  Rearrangement 

The  hyperplanes  characterized  by  S  i  ,Sj . S  y  are 

rearranged  such  that  the  new  hvperplanes 
Sf.Sf . S\  are  orthogonal  to  their  adjacent  hyper  - 


planes.  'I'he  algorithm  to  perform  this  is 

Sf=s,.  .7. 

S,*  =  S„  -  !<S„  'S„.  ,i  (S.  |iiS„  ,. 

n  =  'J.:( . V.  i  si 

i/»(l  i  =  i/i  I).  no 

ii  (n )  =  //(// 1  -  |(S„ 1  S„  , I  < S , '  S.  ,(',//  (ii  -  li. 

II  =  L’.ll . V  I  llll 

Now  the  matrix  equation  to  be  solved  is 

K"a  =  u*.  dll 


where  the  superscript  #  throughout  indicates  modified 
quantities. 

B.  Projection  Algorithm 

The  projection  step  starts  with  an  initial  guess  an  for 
the  solution  vector.  This  point  a,,  in  the  .V-dimeiisiona! 
space  is  then  orthogonally  projected  onto  the  first  hy 
perplane  Sf  1  a  =  u  (1)  to  yield  the  next  solution  vector 
estimate  a|.  In  general  the  projection  onto  the  nth 
hyperplane  S,f  y  a  =  u  Pi)  results  in  the  nth  estimate 
for  the  solution  vector: 
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a„  =a„~i  -  ||a„-i'S;,  -  u'(M)|/lS;,7S;,)|S;t. 

11  =  1,2 . .V.  (121 

Once  a.v  is  determined,  it  is  projected  back  again  onto 
the  first  hyperplane  to  start  a  second  iteration.  This 
process  of  going  from  an  to  a.v  is  known  as  one  complete 
iteration.  It  has  been  observed10  that  this  algorithm 
converges  to  the  true  solution  in  a  small  number  of  it¬ 
erations. 

IV.  Numerical  Results 

As  was  discussed  earlier,  all  three  filters  use  the  same 
matrix  R  for  a  given  input  training  set.  Once  R  is  de¬ 
termined,  the  coefficients  for  the  three  LCFs  can  be 
obtained  by  using  the  MHP  method  in  Eq.  (5)  with  a 
common  matrix  R.  Only  the  constraint  vectors  are 
different  in  the  three  cases. 

As  an  illustrative  example,  we  considered  the  input 
data  set  used  by  Hester  and  Casasent.0  This  data  set 
consists  of  seven  IR  views  of  a  tank.  We  applied  the 
MHP  technique  to  the  7X7  correlation  matrix  R  pro¬ 
vided  by  Hester  and  Casasent.1  To  obtain  the  EOP 
filter  we  used  the  constraint  vector  |1  1  1  1  1  1  1|  1 .  The 


starting  guess  for  the  solution  vector  was  also  a  vector 
with  all  ones,  i.e., 

a„  =  1 1  l  l  i  1  ll| 1 121 

The  recursions  are  carried  out  using  the  MHP  algo¬ 
rithm  outlined  in  Sec.  Ill,  and  these  recursions  are 
stopped  when  the  error, 

err  =  \/  a„  -  a*  .  <  14) 

decreases  below  a  predetermined  threshold,  where 
II  •  ||  denotes  the  Euclidean  norm  of  the  vector  and  a’ 
denotes  the  true  coefficient  vector  obtained  by  the  di¬ 
rect  solution  of  the  corresponding  equation.  In  general 
a*  will  not  he  available  a  priori  and  we  can  use  the  dif¬ 
ference  between  successive  estimates  of  a  to  monitor 
convergence.  The  MHP  can  be  deemed  to  be  successful 
if  the  error  decreases  as  the  number  of  complete  itera¬ 
tions  increase. 

The  error  in  the  coefficient  vector  obtained  by  the 
MHP  method  is  listed  in  Table  I  for  ECP,  MOF,  and 
NRF  linear  combination  filters.  Close  inspection  of 
Table  I  reveals  that  the  solution  vector  obtained  after 
ten  complete  iterations  is  quite  accurate.  After 


Table  I.  The  rms  Errors  V$  Iteration  Number 


Iteration 

no. 

K(T 

MOK.9 

NKF1 

i 

2.2709 

2.0099 

2.0458 

•> 

1.1 9, SO 

0.2111 

1.42.59 

3 

0.4705 

0.0403 

0.5015 

i 

0.1821 

0.0147 

0.2189 

r> 

0.07 137 

0.0058*2 

0.08557 

0 

0.02797 

0.00228 

0.02.244 

*7 

0.01093 

0.00089 

0.01307 

8 

0.00427 

0.00035 

0.00510 

9 

0.00107 

0.00014 

0.00199 

10 

0.00005 

0.00005 

0.00078 

lf> 

0.0  x  ur  • 

0.5  X  I0-‘; 

0.7  X  lo  '■ 

20 

0.7  X  10- ; 

0.4  X  10— 

0.9  X  10-T 

23 

0.1  X  io- 

0.2  X  10- !* 

0.9  X  111-1' 

Table  II.  The  rms  Errors  Vs  Iteration  Number 

Iteration 

no.  .V  =  3  ,V  -  4  A'  =  5  A  =  0  A  =  7 


1 

1.2432 

1.150“ 

1.21  1 

1.2928 

2.3709 

'> 

0.0700 

0.035 

o.l  5020 

0.20394 

1 .1980 

3 

o.o  x  io- 1 

0.18  X  10  - 

0.07  x  Hr  * 

0.01592 

0.4705 

4 

0.55  X  10 

0. 10  X  10  ; 

0.40  X  10  1 

0.97  X  10  ; 

0.18314 

5 

0. 15  X  10  : 

0.14  X  10  ‘ 

0.17  X  10-  * 

0.89  X  10“  1 

0.07150 

0 

0.12  x  10  ' 

0.43  X  10"  * 

0.91  X  10  ' 

0.02797 

7 

0.11  X  10-« 

0.37  X  10 

0.93  X  10  *' 

0.0109 

8 

0.37  X  10  H 

0.38  X  10  ■ 

0.85  X  10 

0.0043 

9 

0.19  X  10  " 

0.1  1  X  10" 7 

0.12  X  10  ■ 

0.0017 

10 

0.85  X  10“H 

0.53  X  10  * 

O.OOO05 

11 

0.1  1  X  10  s 

0.42  Xlll  ' 

0.00025 

12 

0.50  X  10  s 

O.tHHM  19 

13 

0.53  X  JO  H 

0.38  X  10  ‘ 

14 

0.42  X  10  s 

0.15  X  10  1 

15 

0.59  X  10  ’ 

20 

0.09  X  |0  ■ 

•>5 

0.37  x  lo  - 
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twenty-five  complete  iterations,  an  error  of  only  0.4  X 
10~8  is  observed  indicating  an  accurate  solution  using 
the  MHP.  If  an  accuracy  of  0.1%  suffices,  we  need  to 
use  only  ten  complete  iterations  for  the  design  of  an 
EOF  filter. 

In  all  our  numerical  experiments,  we  have  observed 
that  the  rms  error  monotonically  decreases  to  a  small 
value  as  the  number  of  iterations  is  increased  until  a 
certain  stage  beyond  which  the  error  either  remains 
constant  or  oscillates  between  two  small  values.  '1’his 
can  b<>  asily  shown  to  be  due  to  the  finite  accuracy  in 
the  digital  computer  representation. 

The  number  of  iterations  required  to  obtain  a  solution 
vector  of  a  desired  accuracy  depends  on  the  size  of  the 
matrix  R.  To  illustrate  the  dependence  of  convergence 
on  the  size  N  of  the  input  data  set,  we  considered  dif¬ 
ferent  input  data  sets  |Xi,Xj . X.\|,  where  N  can 

take  on  values  from  11  to  7.  The  resulting  correlation 
matrices  are  simply  the  principal  minors  of  the  7X7 
matrix  R.  The  rms  errors  obtained  after  each  iteration 
for  various  matrix  or  training  set  sizes  are  shown  in 
Table  II.  From  this  table  we  see  that  more  iterations 
are  required  to  obtain  the  same  accuracy  as  the  size  N 
of  the  input  data  set  is  increased. 

Thus  far  we  have  discussed  the  use  of  the  MHP  for 
ECP  filter  determination  and  its  convergence  perfor¬ 
mance.  A  similar  numerical  analysis  was  carried  out 
for  the  seven  mutually  orthogonal  filters  (MOF)  as  well 
as  the  three  binary  nonredundant  filters  (NRF)  of  Sec. 
II.  The  convergence  behavior  for  MOF3  as  well  as  for 
NRFl  is  presented  in  Table  I.  MOF3  is  obtained  using 
the  MHP  method  in  a  manner  similar  to  that  used  for 
the  ECP  except  that  a  constraint  vector  |0  0  1  0  0  0  0) 7 
was  used  instead  of  a  vector  with  all  ones.  Similarly, 
NRF  I  was  obtained  by  using  the  constraint  vector  (1  0 
10  10  1] |7'.  We  note  from  Table  I  that  the  rms  error 
once  again  decreases  as  the  number  of  iterations  is  in¬ 
creased.  Less  than  ten  complete  iterations  are  needed 
to  achieve  an  error  <0.1%. 

Strictly  speaking,  only  the  seven  MOFs  need  to  be 
computed.  The  ECP  and  NRF  filters  can  then  be  ex¬ 
pressed  as  simple  known  linear  combinations  of  these 
seven  MOFs.  Such  an  approach  is  computationally 
inefficient  when  only  one  type  of  filter  is  of  interest. 
For  example,  when  an  ECP  filter  is  needed,  the  i  direct 
approach  using  MOFs  requires  seven  times  more  work 
compared  with  the  direct  approach  that  utilizes  the 
constraint  vector  [1  1  1  1  1  1  1|  7  . 


One  can  easily  expand  the  NRF  synthesis  to  an  ar¬ 
bitrary  base  as  8.  With  a  base  of  8,  the  NRF  would  yield 
the  integer  i  as  the  output  when  the  input  is  X,.  Then 
the  constraint  vector  becomes  [l  2  3  4  5  6  7) 7 .  Even 
with  such  filters  we  did  not  need  more  than  ten  com¬ 
plete  iterations  for  an  accurate  solution. 

Thus,  we  have  shown  that  for  various  filter  designs 
we  can  use  MHP  for  less  than  ten  complete  iterations 
when  the  input  set  contains  seven  patterns.  We  have 
also  shown  this  to  be  a  unified  computational  approach 
to  the  design  of  linear  combination  filters  since  only  the 
constraint  vectors  need  to  be  changed  from  one  type  of 
filter  to  another  type. 


V.  Conclusions 

For  multiclass  pattern  recognition,  several  ap¬ 
proaches  utilize  linear  combination  filters  for  improved 
discrimination.  All  these  approaches  are  based  on 
deterministic  constraints  and  can  be  modeled  as  similar 
matrix/vector  problems.  Such  an  approach  unifies  the 
computational  considerations  because  only  the  appro¬ 
priate  constraint  vectors  need  to  be  specified  for  dif¬ 
ferent  filters.  In  this  paper  we  have  examined  the 
convergence  behavior  of  the  modified  hyperplane 
(MHP)  method  in  the  filter  design.  Good  convergence 
properties  were  exhibited  for  the  correlation  matrices 
that  arose  in  these  cases. 
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ABSTRACT 


for  four  different  types  of  SDFs. 


Our  unified  synthetic  discriminant  function 
(SDF)  filter  synthesis  technique  using  the  correla¬ 
tion  matrix  of  the  image  training  set  is  reviewed. 
Four  different  synthetic  discriminant  functions  for 
intra-class  recognition,  inter-class  discrimination 
and  both  intra  and  inter-class  pattern  recognition 
are  considered.  All  techniques  proposed  are  appro¬ 
priate  for  object  identification,  location  and  clas¬ 
sification  in  the  presence  of  3-D  geometrical  dis¬ 
tortions  in  the  input  object.  Initial  results  ob¬ 
tained  on  a  set  of  four  different  classes  of  infra¬ 
red  ship  imagery  are  presented.  Excellent  perform¬ 
ance  (over  90%  correct  classification)  was  achieved. 


1  ■  INTRODUCTION 

Many  different  optical  pattern  recognition 
techniques  to  permit  object  recognition  and  classi¬ 
fication  in  the  face  of  geometrical  distortions 
havq  been  proposed.  These  include:  space-variant 
processors1-2,  generalized  matched  filters1-1*,  syn¬ 
thetic  discriminant  functions  (SDFs)5-7,  phase- 
coded  systems*  and  others.  In  this  paper,  we  con¬ 
sider  new  SDF  research.  The  SDF  approach  to  3-D 
distortion-invariant  pattern  recognition  appears  to 
be  most  attractive,  since  a  matched  spatial  filter 
of  the  SDF  can  be  produced  and  used  in  an  optical 
frequency  plane  correlator.  Thus,  SDFs  exhibit 
shift-invariance  and  hence  are  capable  of  recogniz¬ 
ing  multiple  objects  in  the  field  of  view  and  pro¬ 
viding  information  on  their  location.  Other  tech¬ 
niques  such  as  space-variant  and  phase-coded  sys¬ 
tems  require  the  input  object  to  be  centered  in  the 
input  plane.  Since  SDFs  are  used  in  correlators, 
they  provide  a  processing  gain  and  are  thus  capable 
of  locating  objects  in  the  presence  of  severe  noise 
and  man-made  clutter. 

In  Section  2,  we  describe  SDFs  and  in  Section 
3  we  discuss  our  new  generalized  technique*  for  SDF 
synthesis  and  how  the  same  basic  algebraic  equation 
can  be  used  to  synthesize  four  different  types  of 
SDFs.  This  synthesis  technique  is  attractive  be¬ 
cause  it  automatically  determines  the  optimum  basis 
function  set.  It  is  thus  preferable  to  ad  hoc 
techniques  and  to  sort  restricted  techniques *"* 
using  a  Fourier  transform  coefficient  basis  func¬ 
tion  set.  In  Section  4,  we  discuss  the  infrared 
ship  image  data  base  we  used  and  in  Section  S  we 
present  our  initial  experimental  results  obtained 


2.  SDF  BASIC  SYNTHESIS 


The  general  pattern  recognition  problem  that 
we  initially  consider  is  the  design  of  a  filter 
function  h(x,y)  that  can  recognize  all  objects 
{fn(x,y)  }  of  one  class  of  input  data,  independent 
of  geometrical  distortions  present  in  the  input 
image.  As  noted  in  Section  1,  we  plan  to  optically 
correlate  all  images  {fn}  with  h  and  will  design  h 
such  that  all  correlations  are  equal  to  a  constant 
output  value,  i.e. 

h  ©  fn  -  c.  (1) 

As  our  constant  c,  we  select  unity  (with  no  loss  of 
generalization) .  Because  this  technique  is  quite 
analogous  to  a  hyperspace  description  of  the  input 
image  data,  we  represent  input  images  and  the  SDF 
h  by  vectors  and  write  (1)  as 

h  •  in  “  1.  (2) 


In  conventional  pattern  recognition  terminolo¬ 
gy,  one  would  describe  each  fn  as  a  linear  combina¬ 
tion  of  a  set  of  basis  functions  ifj ,  i.e. 


In  *  ?anjlj ' 


(3) 


where  in  (3)  the  fn  and  are  2-D  functions  of  x 
and  y  (i.e.  images).  We  write  each  in  terms  of 
a  linear  summation  of  the  input  images  fn  as 


£j  "  ndinln-  14  * 

Analogous  to  (3),  we  write  our  unknown  filter  func¬ 
tion  1  as  a  linear  combination  of  the  same  basis 
functions 


h  - 

Substituting  (4)  into  (S) ,  we  obtain 

a.  ■  pj^min 

*  bl£dlnln  +  b2nd2n£n  +  • ■ • 


(5) 


(6) 


Grouping  terms  in  fj  and  f^,  etc.  separately,  we 
can  rewrite  (6)  as 


4  m2*3  * 


fi'nln’ 


(7) 


We  have  now  described  our  filter  function  h  as 
a  linear  combination  of  the  input  training  set  (fnl. 
It  remains  only  to  determine  the  weighting  coeffi¬ 
cients  en  in  (7)  and  then  we  have  determined  our 
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MICROCOPY  RESOLUTION  TEST  CHART 


SOP  h.  To  achieve  this,  «•  rscsll  our  condition  In 
(2)  and  substitute  (?)  into  (2)  to  obtain 

fin  "  1  "  <J«  j* j)  •  in  -  ^aj  (f i  •  f„) .  (8) 

Na  recognize  the  last  ter*  in  parentheses  in  (8)  to 
be  the  correlation,  or  projection,  of  f j  onto  fn. 
This  is  element  R,n  of  the  corr'lation  matrix  R  of 
the  training  set  data  {fn)  and  (8)  thus  becomes 

5«jRjn  “  l-  (9> 

He  write  (9)  in  matrix-vector  notation  as 

Re  ■  u,  (10) 

where  u  is  the  unit  vector.  The  solution  e  to  (10) 
and  hence  the  information  necessary  to  calculate 
the  SDF  h  in  (7)  is 

e  -  R_1u.  (11) 

Thus,  to  determine  the  e„  in  (9)  or  the  eigenvalue 
in  (11)  ,  we  form  the  correlation  matrix  R  of  the 
training  set  data  {fn)>  invert  this  matrix  and  mul¬ 
tiply  it  by  the  constant  vector  u. 

3.  GENERALIZED  SDF  SYNTHESIS 

He  refer  to  the  SOP  described  in  Section  3  as 
an  equal  correlation  peak  (ECP)  SDF,  since  it  pro¬ 
vides  a  constant  equal  correlation  peak  value  for 
all  input  isiages  {fn)  of  one  class  of  data.  We 
should  note  explicitly  that  the  images  in  the  train¬ 
ing  set  (fn)  can  (and  usually  are)  images  of  the 
same  object  taken  with  different  geometrical  dis¬ 
tortions  (e.g.  different  rotated  or  aspect  views) . 
This  type  of  SDF  is  of  use  in  intra-class  pattern 
recognition  (recognition  of  one  object  class,  inde¬ 
pendent  of  geometrical  distortions  within  this  ob¬ 
ject  class) .  This  is  similar  to  our  initial  SDF 
work',  but  uses  a  new  SDF  synthesis  technique. 

Next,  we  consider  a  mutual  orthogonal  function 
(HOF)  SDF  for  inter-class  pattern  recognition.  We 
consider  a  two-class  pattern  recognition  problem 
with  one  image  fj  in  class  one  and  one  image  f2  in 
class  two.  We  desire  two  filters  h2  and  h2,  such 
that 


We  now  provide  a  generalization  of  this  tech¬ 
nique  that  allows  both  intra-class  and  inter-class 
discrimination.  We  formulae  this  new  HOF  SDF  for 
the  case  of  two  object  classes  with  N}  and  Nj  images 
respectively  in  each  class.  We  desire  two  SDFs  hj 
and  hj,  auch  that 

»U  *  Ini  ’  hi  *  In2  “  0 

(16) 

h2  '  L.2  "  0,  !l2  *  In2  *  I* 


where  f^j  and  f^2  denote  any  member  of  class  one  or 
class  two  respectively.  We  describe  these  SDFs  as 
in  (14) ,  where  the  sumnation  is  now  over  Nj  ♦  n2 . 
There  are  now  Ni  +  Nj  weighting  coefficients  an  and 
bp.  They  are  given  by  (15) ,  where  now  R  is  an 
(Ni  +  Nj)x(Ni  ♦  N2)  matrix  and  ujT  -  IlT.  .1 ,0. .  .0] 

contains  Nj  ones  and  N2  zeroes  and  U2T  -  [0 0, 

1...1]  contains  Nj  zeroes  and  N2  ones.  We  refer 
to  this  as  a  mutual  orthogonal  correlation  peak 
(MOCP)  SDF.  It  recognizes  different  object  classes 
(inter-class  discrimination)  even  in  the  face  of 
geometrical  distortions  within  each  object  class 
(intra-class  recognition) .  We  will  discuss  later  how 
this  technique  can  be  extended  to  greater  than  two 
classes.  We  also  note  that  for  the  two  object 
class  example,  one  filter  suffices  for  both  dis¬ 
crimination  and  recognition.  We  also  note  that  the 
choice  of  the  constants  in  (13)  and  (16)  is  arbi¬ 
trary.  In  our  digital  simulations,  we  used  +1, 
rather  than  1  and  0.  An  optical  system  would  use 
0  and  1  or  1  and  2,  etc.,  since  only  intensity  out¬ 
put  detection  is  easily  possible  in  optical  proces¬ 
sors. 


Next,  we  consider  the  synthesis  of  a  no  nr ed un¬ 
dent  SDF  or  a  nonredundant  filter  (NRP) .  This  is 
similar  to  those  described  by  Braunecker  et  al  “  ex¬ 
pressed  in  terms  of  our  unified  notation  in  (11) 
and  (15) .  We  also  extend  the  original  work  in”  * 
the  case  of  a  NRF  for  both  intra-class  recognit j ,n 
and  inter-class  discrimination.  As  an  example,  we 
consider  a  three-class  pattern  recognition  problem. 
We  denote  the  three  classes  of  objects  by  ( ffji ) , 
{f„2)  And  { f N3 ^ •  where  Nj,  N2  and  N3  training  set 
images  are  present  respectively  in  each  object 
class.  We  consider  one  SDF  h,  such  that 


In  '  h«n  “  ^nm'  ^2) 

i.e. 

fi*hi-i,  fa'hjwo 

(13) 

fl  •  hj  ■  0,  •  hj  ■  i. 

Each  filter  will  again  be  a  linear  combination  of 
all  training  set  images  (fi  and  f2> ,  i.e. 

hi  -  &»n*n*  h2  -  *bn<n-  U«> 


To  compute  the  a„  and  the  bn  coefficients  in  (14) , 
we  solve  the  matrix-vector  equations 

R a  •  uj,  R b  ■  jj2  ,  (15) 

where  R  is  the  2x2  correlation  matrix,  u^?  - 
(1,0)  and  u?T  ”  (0,1).  This  filter  synthesis  tech¬ 
nique  is  similar  to  that  described  in* ,  but  is 
formulated  in  our  new  unified  notation.  This  tech¬ 
nique  produces  one  filter  per  input  object  class. 

As  such,  it  is  useful  for  inter-class  pattern  rec¬ 
ognition. 


H  '  Ini  -  t  *  ln2  ’  2-  H  ’  in3  '  3  (17> 

where  as  before,  fnj,  fj,2  and  f,^  denote  any  member 
of  object  class  one,  two  or  three  respectively.  As 
seen,  the  value  of  the  output  correlation  deter¬ 
mines  the  object  class.  As  before,  the  choice  of 
the  constants  1,  2  and  3  in  (17)  is  arbitrary.  We 
describe  the  SDF  h  as 

h  -  la,^,  (18) 

where  the  sumnation  is  over  all  Nj  +  N2  ♦  N3  train¬ 
ing  set  images.  In  matrix-vector  terminology,  the 
problem  is  described  as  the  solution  of  R  a  «  u_i  for 
a,  where  U3  •  (1. . .1,2. . . .2,3. . .3)T  contains  n3 
ones,  N2  twos  and  Nj  threes  and  where  R  is  of  order 
Ni  +  N2  +  H3.  The  solution  for  the  weight  a  in 
(18)  is  just  a  -  R“ luj  as  before. 

As  our  final  SDF,  we  consider  a  multi-class 
MOCP  SDF.  For  this  case,  we  consider  a  four-class 
problem  with  training  set  images  ( fpi } ,  {f^},  tfN^ 
and  {fjjf}  and  the  synthesis  of  two  filters  whose 
truth  table  is  given  in  Table  1.  Pram  Table  1,  we 
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■ec  that  four  unique  pair*  of  correlation  output 
values  result  and  determine  the  input  object  class. 
The  two  filters  are  described  by 


ti '  £Vn* 


Ebf  . 

n-r, 


where  the  summations  are  over  ♦  N2  +  Nj  ♦  N4 
(i.e.  the  full  training  set  of  images).  The  coef¬ 
ficients  a  and  b  are  defined  by  the  solution  to  the 
aiatrix-vector  equation 


where  the  vector  has  Ni  pairs  (0,0),  N.  pairs  (0,11 , 
etc.  and  where  R  is  of  order  ♦  Nj  +  N3  ♦  N4. 

TABLE  1 

Truth  Table  for  the  Four-Class 
Multi-Class  MOCP  SDF 


FILTER  hx  FILTER  h2 
OUTPUT  OUTPUT 


4  ■  DATA  BASE 

To  test  the  performance  of  the  four  SDFs  de¬ 
scribed  in  Section  3,  we  used  a  ship  image  data 
base.  It  contained  36  images  (0-350*,  at  10*  in¬ 
crements)  of  four  different  ships  taken  from  a  90* 
depression  angle.  Representative  examples  of  all 
four  ships  (broadside  views  only)  are  shown  in  Fig¬ 
ure  1.  Each  image  was  128  x  32  pixels.  Me  numbered 
the  class  one  images  one  1-36,  the  class  two  images 
as  37-72,  etc. 


b)  CLASS  2 


d)  CLASS  4 

FIGURE  1  Representative  Ship  Imagery  (Broadside 
View  Only)  of  the  Ship  Image  Data  Base 
Used  in  our  Experiments. 


5.  INITIAL  SIMULATED  RESULTS 

In  Table  2,  we  summarize  the  experimental  re¬ 
sults  for  our  four  SDFs  calculated  as  described  in 
Section  3. 

For  our  ECP-1  SDF,  we  calculated  the  intra¬ 
class  filter  from  only  six  of  the  36  ship  images  in 
class  one.  When  this  SDF  was  correlated  with  all 
36  images  of  class  one,  all  output  correlation  peak 
intensities  were  above  the  0.5  threshold  value  cho¬ 
sen.  Similar  results  were  obtained  for  the  class 
two  imagery .  This  data  is  contained  in  the  ECP-2 
SDF  experiments  in  Table  2.  These  first  two  ex¬ 
periments  desionstrated  for  the  first  time  the  fact 
that  an  ECP  SDF  can  correctly  recognise  and  classi¬ 
fy  other  distorted  versions  of  the  input  object  not 
present  in  the  training  set. 

For  our  MOCP  SDF,  we  calculated  one  SDF  for 
intra  and  inter-class  pattern  recognition,  using 
the  same  six  training  set  images  per  class.  For 
this  one  SDF,  69  of  the  72  test  images  gave  cor¬ 
rect  peak  correlation  outputs  with  respect  to  the 
threshold  of  0.5.  This  represents  over  97t  correct 


TABLE  2.  SDF  Performance  Test  Results 


SDF 

TYPE 

TRAINING 

SET 

c 

1 

LASS  I, 
ERRORS 

CLASS  2 
ERRORS 

CLASS  3 
ERPORS 

CLASS  4 

ERRORS 

ECP-1 

ECP-2 

1,6,10,15, 

20,25 

38,45,50, 

55,60,65 

0 

0 

- 

- 

KOCP-1 

<AP*0.5> 

1,6,10,15, 

20,25 

1 

2 

- 

- 

NR F 

(6  Images 
Per  Class) 

2 

0 

3 

- 

Multi- 

Class 

M0CP-1 

<il> 

(6  Images 
Per  Class) 
(10*-340* ) 

2 

(95%) 

0 

(100%) 

3 

(92t) 

1 

(97t> 

Multi- 
Class 
MOCP -2 

U 1) 

(6  Images 
Per  Class) 
(10* -260*) 

0 

(100») 

0 

(loot) 

4 

(97t) 

5 

<96t) 

recognition.  This  is  quite  impressive  since  it  in¬ 
cludes  both  intra  and  inter-class  image  distortion 
and  recognition  as  well  as  discrimination. 


Our  MB’  SDT  experimental  results,  for  a  three- 
class  pattern  recognition  problem,  using  output 
correlation  peak  projection  values  of  (0,1,2)  are 
shown  next,  ha  noted,  only  5  errors  occurred  in 
this  case.  When  this  MRP  SDF  technique  was  exten¬ 
ded  to  the  case  of  four  object  classes,  a  consid¬ 
erable  increase  in  the  maber  of  image  classifica¬ 
tion  errors  resulted. 

Encouraged  by  the  above  excellent  results,  we 
considered  the  full  four-class  pattern  recognition 
problem  using  two  SDFs  (our  multi-class  MOCP  SDF) . 
We  fabricated  one  set  of  SDFs  using  9  images  per 
class  for  training  and  a  second  set  of  SDFs  using 
only  6  images  per  class  for  the  training.  As  seen, 
a  total  of  6  errors  (and  9  errors)  out  of  144  im¬ 
ages  resulted.  These  correspond  to  over  934 
correct  recognition  of  all  multi-class  objects  with 
intra-class  geometrical  distortions  present. 

6.  SUMMARY  AND  CONCLUSION 


These  initial  results  are  most  encouraging. 
They  demonstrate  the  potential  of  this  technique  to 
achieve  intra-class  pattern  recognition  and  inter - 
class  diserbnination  in  the  presence  of  severe 
(3-D  aspect)  geometrical  distortions.  The  data 
shown  clearly  indicates  the  ability  of  this  tech¬ 
nique  to  perform  most  excellently  on  images  not 
present  in  the  image  training  set.  Our  future  work 
will  address  training  set  selection  and  the  perform¬ 
ance  of  the  system  in  the  presence  of  noise.  Our 
unified  SDF  synthesis  technique  greatly  simplifies 
the  off-line  calculations  required.  The  fact  that 
one  can  calculate  the  SDF  off-line  from  training 
set  images  and  then  use  the  same  SDF  for  similar 
abject  recognition  problems  in  the  future  is  of 
great  significance  in  the  practical  application  of 
such  a  technique. 
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ABSTRACT 

The  problem  of  recognizing  multiple  objects  in  a  highly  cluttered  background  in  the  face 
of  geometrical  object  distortions  is  addressed  in  this  paper.  A  correlation  architecture 
using  a  matched  spatial  filter  of  a  synthetic  discriminant  function  is  employed  to  achieve 
the  required  performance.  Synthesis  of  the  synthetic  discriminant  function  is  discussed  as 
is  the  initial  performance  obtained  in  the  face  of  noise.  Initial  remarks  are  advanced  on 
various  methods  to  select  the  training  set  of  images  to  use  in  this  algorithm. 

1,  INTRODUCTION 

In  Section  2,  the  advantages  and  features  of  a  correlator  for  extracting  a  signal  or  ob¬ 
ject  in  the  presence  of  noise  is  briefly  reviewed.  A  correlator  is  essential  for  our  par¬ 
ticular  application  since  we  require  shift-invariance  (the  ability  to  recognize  multiple  ob¬ 
jects  and  obtain  high  performance  in  the  face  of  severe  noise  or  clutter).  However,  corre¬ 
lators  are  well-known  to  perform  quite  poorly  when  geomtrical  distortions  (scale,  rotation, 
etc.)  are  present  between  the  input  and  reference  object.  To  overcome  this  limitation  and 
still  retain  the  other  advantageous  features  of  a  correlator,  we  employ  synthetic  discrimi¬ 
nant  functions  (SDFs)  and  we  form  the  matched  spatial  filter  (MSF)  from  the  SDF.  The  EDF  is 
a  linear  combination  of  a  training  set  of  images  of  the  object  (e.g.  different  scaled,  rota¬ 
ted  or  aspect  views).  This  concept  was  first  advanced  in  [1-3].  Other  MSFs  referred  to  as 
generalized  matched  filters  [4]  also  exist.  These  assume  a  Fourier  coefficient  basis  func¬ 
tion  set  and  appear  to  be  more  restrictive  than  the  present  filters  we  discuss,  although  no 
extensive  tests  have  been  performed  on  them. 

In  Section  3,  we  describe  the  algorithm  for  synthesis  of  five  different  types  of  SDFs. 

The  SDF  is  synthesized  off-line  from  training  set  data  and  is  then  used  in  a  real-time  cor¬ 
relator  against  test  imagery  not  present  in  the  training  set.  We  follow  the  new  unified  SDF 
synthesis  algorithm  we  briefly  described  in  [5]  earlier.  In  Section  4,  we  discuss  our  data 
base  used  and  we  present  initial  test  results  on  the  performance  of  these  SDFs.  These 
present  results  include  new  data  on  the  performance  of  such  a  system  in  the  presence  of 
noise.  Our  earlier  initial  and  less  extensive  results  [6]  did  not  consider  the  performance 
of  such  a  system  ir.  the  presence  of  noise.  In  Section  5,  we  advance  initial  remarks  on  two 
new  techniques  fc r  training  set  selection  and  the  superior  performance  obtained  with  the  use 
of  an  organized  rather  than  an  ad  hoc  training  set  selection  method.  The  bulk  of  the  test 
data  included  ir.  Section  4  uses  an  ad  hoc  training  set  selection  technique  (to  allow  more 
errors  than  one  could  otherwise  achieve)  and  the  use  of  a  "no  decision”  threshold  region  (to 
stabilize  statistical  fluctuations  in  the  results  obtained). 

2.  CORRELATORS  FOR  PATTERN  RECOGNITION 

Correlators  are  very  well-known  [7]  to  be  ciuite  capable  of  extracting  signals  or  objects 
from  a  cluttered  background  environment.  A  MSF  correlator  is  optimal  for  Gaussian  white 
noise  backgrounds,  but  performs  most  admirably  in  the  face  of  structured  noise  as  well. 
Whitening  of  the  image  and  noise  data  is  possible  and  can  allow  optimal  performance,  but 
such  measures  do  not  appear  to  be  necessary  in  the  present  problems  and  applications  being 
considered.  Correlators  are  also  well-known  to  be  linear  shift-invariant  systems.  As  such, 
they  allow  multiple  objects  to  be  recognized  within  the  input  field  of  view. 


FIGURE  1.  Block  diagram,  of  a  Fourier  Transform-Based  Correlator  Architecture. 
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The  block  diagram  of  the  basic  Fourier  transform  (FT)  based  correlator  is  shown  in  Fiaur 
1.  The  system's  output  is 

u  (x ,  y )  =5fcrllG*F)  =  g  ©  f  (1, 

or  the  correlation  of  the  two  space  functions  g  and  f  (G  and  F  denote  the  Fourier  transform 
of  the  corresponding  space  functions).  The  processing  gain  (PG)  of  a  correlator  (the  ratio 
of  the  output  and  input  signal  to  noise  ratio  SNRq/SNRi)  is  related  to  the  space  bandwidth 
product  (SBWP)  of  the  MSF  by 

PG  =  (SBWP)  =  SNPp/SNR  .  (2) 

This  PG  allows  a  correlator  to  extract  objects  from  strong  cluttered  background.  The  shift 
invariant,  multiple-object  recognition  ability  and  the  processing  gain  of  a  correlator  are 
easily  demonstrated  by  considering  the  general  case  of  an  input  image  f(x,y)  that  contains 
N  occurrences  of  the  reference  object  g  at  spatial  locations  (xn,yn)  plus  noise  n(x,y),  i.e 

f  =  Ig(x-xn,y-yn)  +  r.(x,y).  (3) 

The  output  correlation  for  this  case  consists  of  N  autocorrelations  plus  the  cross  cor¬ 
relation  of  the  reference  object  and  the  noise,  i.e. 

out  =  f  ©  g  =  g©g*-‘  (x-xn  ,y-yn>  +  g  ©  n .  U  ) 


(a)  Input  Image 


(b)  Output  Correlation  of  (a) 
and  the  word  PROFErSOR. 


FIGURE  2.  Example  of  multiple-object  shift-invariant  correlation. 

An  example  that  vividly  demonstrates  this  point  is  shown  in  Figure  2,  where  we  show  the 
input  image  f  (a  set  of  words  with  four  occurrences  of  the  word  PROFESSOR)  and  the  output 
correlation  plane  pattern  obtained  using  a  MSF  of  the  word  PROFESSOR.  As  seen,  four  corre¬ 
lation  peaks  are  present  and  the  location  of  each  corresponds  to  the  location  of  one  of  the 
different  occurrences  of  the  reference  object  (here  the  word  PROFESSOR)  in  the  input  plane. 
All  other  input  words  in  this  input  image  are  viewed  as  the  noise  n(x,y)  in  (3)  and  (4)  for 
this  case.  As  seen,  the  correlation  output  plane  has  a  very  low  noise  level.  This  is  due 
to  the  large  SBWP  of  the  word  PROFESSOR  and  the  PG  of  a  correlator. 

3.  SYNTHETIC  DISCRIMINANT  FUNCTION  SYNTHESIS 


The  general  concept  of  SDF  synthesis  is  shown  in  the  simple  block  diagram  of  Figure  3.  A1- 
SDF  is  calculated  off-line  from  training  set  images  fy,  f2,  etc.  and  then  on-line  correlate 
with  a  real  test  image.  The  preprocessing  box  shown  is  generally  not  necessary  as  noted 
above.  A  hyperspace  description  of  a  SDF  correlator  is  possible  12-3).  In  such  a  formula¬ 
tion,  each  training  set  image  is  described  by  a  vector  point  in  hyperspace  with  the  basis 
functions  for  this  hyperspace  being  2-D  space  functions.  The  conventional  hyperspace  pat¬ 
tern  recognition  concepts  (8]  of  separating  objects  of  different  classes  and  clusterina  dif 
ferent  object  variations  within  a  class  still  apply. 


INPUT® 

IMAGE 


*  TARGET 
IDENTIFICATION 
TARGET 
LOCATION 


FIGURE  3.  Block  diagram  to  illustrate  the  synthetic 
discriminant  function  synthesis  concept 
ar.d  its  use  in  a  correlator. 


3.1  EQUAL  CORRELATION  PEAK  SYNTHETIC  DISCRIMINANT  FUNCTION 


Let  us  now  describe  SDF  synthesis  using  our  new  general  formulation  technique  15).  Ke 
first  consider  an  equal  correlation  peak  (ECP)  SDF.  For  this  case,  we  assume  that  we  wish 
to  determine  an  MSF  filter  function  h  such  that  its  correlation  with  any  member  of  a  set 
{fn}  of  objects  of  the  class  yields  a  constant  output  peak  value  such  as  one,  i.e. 

h  ©  fn  =  1.  (5) 

To  determine  h  to  satisfy  (5) ,  we  assume  that  h  can  be  represented  as  a  linear  combination 
of  the  training  set  images  f,  i.e. 


h  -  Ze.f..  (6) 

3  3  3 

For  simplicity  of  notation,  we  denote  all  images  and  the  MSF  by  vectors  f  or  h.  The  train¬ 
ing  set  images  can  be  different  scaled,  rotated  or  aspect  views  of  the  object.  We  also 
represent  the  correlation  operation  by  the  simple  vector  dot  or  inner  product,  i.e.  h©  f  = 
h  •  f.  Substituting  (6)  into  (5),  grouping  the  dot  product  of  f  and  defining  the  correla¬ 
tion  matrix  R  to  have  elements  rjn,  we  obtain  for  (5) 


3l(G3fi' 


V 


Ee  .r  . 

j  3  Dn 


( . ) 


Thus,  our  desired  ECF  filter  is  described  by 


or  in  matrix-vector  form  by 


g6 jr jn 


1, 


(6) 


Re  *=  u,  (9) 

T 

where  u  «  [1,1,..., 1]  is  the  unit  vector.  The  solution  to  (9)  for  the  weighting  coeffi¬ 
cients  e  in  (6)  is 

£  -  Jf  1u,  (10) 


where  the  ej  in  (6)  are  the  elements  of  the  vector  e  in  (10).  Thus,  to  synthesize  an  ECP 
SDF,  we  form  the  correlation  matrix  of  the  training“set  of  images,  invert  this  matrix  and 
multiply  it  by  the  unit  vector  u.  The  resultant  vector  e  defines  our  SDF  filter  function  h 
in  (6).  To  unify  future  notation,  we  write  (9)  as 


Re  «  [1,1,...,1]T. 


(11) 


3.2  MUTUAL  ORTHOGONAL  FUNCTION  SDF  (2-CLASS,  ONE  FILTER  CASE) 


We  note  that  this  ECP  SDF  MSF  in  (9),  (10)  or  (11)  is  capable  of  intra-class  recognition 
only.  To  extend  this  concept  to  the  use  of  one  filter  for  a  two-class  problem  with  inter- 


class  discrimination  and  intra-class  recognition  required,  we  consider  two  classes  of  objects 
(f3)  and  {f2 },  w’th  Nj  and  N2  objects  in  the  training  set  for  each  class  respectively.  The 

required  SDF  is  „ow  described  by  a  linear  weighted  sum  of  all  of  the  training  set  images, 

i.e. 

N1+N2 

h  =  £  a  f  ,  (12) 

n=l  n  n 

To  determine  the  coefficients  an  in  (12)  ,  we  require  the  projections  of  any  member  fni  of 
set  1  on  h  to  be  +1  and  the  projections  of  any  member  fn2  of  set  two  of  images  to  be  -1 

(any  other  suitable  pair  of  constants  may  be  used) .  By  direct  extension  of  the  results  in 

Section  3.1,  this  desired  mutual  orthogonal  function  (MOF)  SDF  is  defined  by  the  solution  to 
the  matrix-vector  problem 


R  a  —  [1,. ..,1,0,. 


■  .01  =  ux  , 


where  R  is  now  a  (N3+N2 )  x  (N1+N2 )  correlation  matrix  or  is  of  size  2N  x  2N  (when  Nj  =  N2  =  N). 

The  solution  and  the  SDF  in  (12)  is  defined  by 

a  =  R  ^u^ .  (14) 

This  single  MOF  SDF  achieves  intra  and  inter-class  pattern  recognition. 

3.3  NONREDUNDANT  FILTER  SYNTHETIC  DISCRIMINANT  FUNCTIONS  (ONE  FILTER  FOR  MULTI -CLASSES ) 

This  type  of  SDF  is  referred  to  as  a  nonredundant  filter  (NRF)  SDF.  It  is  a  rather  di¬ 
rect  extension  of  the  MOF  SDF  in  (13).  Here,  we  simply  require  the  correlation  output  peak 
intensity  to  equal  a  different  constant  value  (e.g.,  1,  2,  3,  etc.)  for  each  object  class. 
We  consider  a  3-class  pattern  recognition  problem.  The  SDF  is  now  described  by 

n1+n2+n3 


where  the  summation  is  over  all  Ni+N’2+N3  members  of  the  three  training  sets  for  the  three 
object  classes.  For  this  SDF,  the  output  correlation  peak  value  defines  the  input  object 
class.  The  coefficients  an  that  define  this  NRF  SDF  are  the  solution  of  the  matrix-vector 
problem 


[1,...,1,0,...,2,3,...,3]  —  u  ^  * 


That  is, 


a  =  R  u  j ,  ( 1  ' ) 

where  now  R  is  a  3N  x3N  correlation  matrix  (when  Nj  = N2  = N3  =  N). 

3.4  MULTI-CLASS  MOF  SDF  (ONE  FILTER  PER  OBJECT  CLASS) 

In  some  image  pattern  recognition  problems,  the  constraint  placed  on  the  MOF  or  the  NRF 
SDFs  in  Sections  3.2  and  3.3  may  be  too  stringent  and  hence  performance  on  test  data  may  be 
degraded.  In  this  case,  M  separate  SDFs  can  be  produced,  each  of  which  gives  a  constant 
output  value  of  1  for  a  different  object  class  and  a  0  output  for  all  other  object  classes. 
For  a  3-class  pattern  recognition  problem,  we  consider  the  synthesis  of  three  SDF  filters 

h.  -  £a  f  ,  h,  -  Zb  f  ,  and  h,  =  Icf  ,  (18) 

1  n  n  n  2  n  n  n  3  n  n  n 

where  all  summations  in  (18)  are  over  all  Nj+  N2  +  N3  «=  3N  image  training  sets  of  data. 

The  algorithm  for  synthesis  of  these  3  SDFs  is  similar  to  before,  i.e. 

Ra  *  u^  *  (1  •  •  •  1 , 0 •  •  *0 , 0 •  •  •  0]^ ,  a  *  R  ^u^  — h^  .  (19a 

Rb  «  Uj  *  (0 •  •  •  0 , 1  •  •  *1 , 0 •  • ' 03T ,  b  “  R  (19t 

Rc  ■  u3  »  [0*  •  *0,0*  •  *0,1*  •  *1]^,  c  ■  R  ^u3 

In  this  case,  the  correlation  matrix  is  3N  x 3N  (for  Nj  «N2  “N3  ^N) .  By  inspection  of  (19), 
we  see  that  h^  is  selected  to  provide  an  output  of  1  for  the  first  Nj  training  set  images 
(i.e.  class-one  objects)  and  "0"  for  the  other  images  (classes  two  and  three  objects). 


^  ••  d 


Conversely,  SDF  h2  is  defined  to  produce  a  1  output  for  the  N2  training  set  images  in  class 
two  and  a  0  output  for  the  other  classes.  The  class  three  SDF  h3  is  described  similarly. 

3.5  MULTI-CLASS  MOF  SDF  (N  FILTERS  FOR  2N  CLASS  RECOGNITION  PROBLEM) 

For  this  SDF,  we  consider  anot.ier  multi-class  type  of  SDF  formulation  that  is  between  the 
single  SDF  cases  considered  in  Sections  3.2  and  3.3  and  the  multiple  filter  example  considered 
in  Section  3.4.  This  particula  application  is  most  attractive  for  multi-class  pattern  rec¬ 
ognition  problems  in  which  very  many  multiple  classes  of  data  must  be  considered.  Here,  we 
consider  an  M-class  pattern  recognition  problem.  We  assume  binary  correlation  peak  threshold 
outputs  and  the  use  of  N  SDFs  (where  2N  >_  M) .  For  a  four-class  problem,  we  thus  consider  the 
use  of  two  SDFs,  hi  and  h2-  Each  input  image  is  projected  onto  both  SDFs  and  the  two  corre¬ 
lation  outputs  are  used  to  form  a  2-bit  digital  word  which  defines  by  the  truth  table  in 
Table  1  below  in  which  of  the  four  input  classes  ff3)  through  { f 4 1  the  input  object  lies. 

TABLE  1:  Truth  Table  for  Correlation  Peak  Output  Decoding  Using  a  Multi- 
Class  Mutual  Orthogonal  Function  Synthetic  Discriminant  Function 
with  N  Filters  for  a  2N  =M  Class  Pattern  Recognition  Problem. 

M  =  4  and  N  =2  Case  Shown. 


INPUTS 

OUTPUTS 
hi  h2 

<fl) 

0  0 

{f2 } 

0  1 

<f3) 

1  0 

<f4> 

1  1 

The  algorithm  for  calculation  of  the  two  filters  is 


la_f 


Ebf  . 

n  n  n 


(20) 


The  solution  for  the  vectors  a  and  b  that  define  these  two  filter  functions  follows  by  ex¬ 
tension  of  our  previous  SDFs  to  be 
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3.6  UNIFIED  FORMULATION 


By  inspection  of  (11),  (13),  (IS),  (19)  and  (21),  we  find  that  the  synthesis  of  all  five 

SDFs  can  be  described  by  the  same  basic  matrix-vector  equation  R  a  =  u„ ,  where  the  sire  of 
the  correlation  matrix  and  the  specific  form  of  the  vector  Uj,  are' different  for  each  filter 

function  and  for  each  class  of  problem  being  addressed.  This  is  especially  attractive  since 

all  SDF  filter  synthesis  is  performed  off-line.  Thus,  a  simple  digital  matrix-vector  routine 
can  be  used  for  all  cases.  Because  this  SDF  algorithm  relies  on  a  training  set,  the  same 

SDF  filter  will  perform  adequately  until  the  structure  of  the  object  to  be  recognized  changes 

significantly  (due  to  new  designs,  etc.)  or  until  rejection  of  new  alternate  objects  is  re¬ 
quired  (this  depends  on  the  specific  details  of  each  application) .  The  unified  formulation 
in  which  the  solution  of  the  same  basic  matrix-vector  eauation  is  reauired  makes  this  gen¬ 
eralized  formulation  of  SDF  synthesis  most  attractive  [5]  for  many  practical  reasons  and 
applications,  in  all  cases,  if  F  is  not  invertible,  a  generalized  inverse  is  used. 

4.  INITIAL  TEST  AND  PERFORMANCE  RESULTS 

The  data  base  we  used  in  our  test  and  evaluation  of  these  SDF  algorithms  consisted  of 
four  different  types  of  ships.  For  each  ship,  36  images  were  available  at  10°  inteivals 
from  a  0'  depression  angle.  In  Figure  4,  the  broadside  views  of  each  ship  are  shown.  In  Fig¬ 
ure  5,  the  views  at  several  different  aspect  angles  are  shown  for  one  of  the  ships.  From 


this  data,  one  can  clearly  see  the  severe  magnitude  of  the  intra  and  inter -class  recognition 
problem  being  considered.  A  human  cannot  correctly  classify  many  of  the  different  ship  as¬ 
pect  views  from  such  imagery.  Quite  si, ply,  several  of  the  ships  look  essentially  identical 
from  several  different  aspect  views.  Recognition  of  the  ship  class  from  the  bow  or  stern 
views  alone  (e.g.,  Figure  5d)  is  clearly  quite  difficult,  even  for  a  human  observer. 


b)  CLASS  2  d)  CLASS  4 


FIGURE  4.  Representative  ship  imagery  (broadside  view  only) 

of  the  ship  image  data  base  used  in  our  experiments. 


(d)  0'  ANGLE 

FIGURE  5.  Selected  aspect  views  of  one  ship. 


The  results  of  our  initial  experiments  on  four  of  the  different  types  of  SDFs  described 
in  Section  3  are  summarized  in  Table  2.  As  seen,  the  results  obtained  are  quite  excellent 
(over  9SL  correct  recognition  was  obtained  in  all  cases).  The  data  in  Table  2  was  obtained 
with  the  hull  present  and  was  obtained  for  binarized  imagery  properly  thresholded.  It  should 
be  noted  that  in  no  test  were  more  than  9  images  per  class  used  for  training  and  that  in  most 
instances,  only  6  images  per  class  were  used  for  training  and  for  SDF  synthesis. 

In  Table  3,  new  results  obtained  with  different  levels  of  input  noise  added  to  the  test 
imagery  and  to  the  training  set  imagery  are  shown.  To  stabilize  the  results  obtained  and  to 
decrease  fluctuations  in  these  results  due  to  statistical  variations,  any  correlation  peak 
value  that  was  within  ±0.03  of  the  threshold  value  of  0.5  was  classified  as  a  "no  decision" 
image.  The  number  of  no  decisions  obtained  for  this  data  are  entered  in  parentheses,  to¬ 
gether  with  the  number  of  errors  obtained  for  each  class  of  object.  The  hull  of  the  ship 
was  again  retained  in  this  data  and  only  6-9  images  per  class  were  used  for  training.  As 
seen  from  the  last  column  of  Table  3,  the  number  of  correct  recognitions  was  very  constant 
(127  or  128  correct  recognitions  out  of  144  test  images)  with  only  5-9  errors  (and  over  93% 
correct  recognition  in  all  cases) .  The  constancy  of  the  number  of  correct  recognitions  as 
the  input  noise  was  increased  (down  to  an  SNRi  *  6,  corresponding  to  an  input  noise  with  a 
variance  «  0.04)  and  its  invariance  as  different  amounts  of  noise  were  added  to  the  training 
set  itself  indicates  the  robustness  of  this  SDF  filter  to  noise.  Since  this  is  a  correlator, 
we  expect  such  performance  as  noted  in  Section  2.  Tests  in  which  noise  was  present  in  the 
background  only  and  in  both  the  background  and  the  target  were  also  conducted.  Results  very 


similar  to  those  obtained  in  Table  3  again  resulted  for  this  case. 
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TABLE  2.  Initial  Noise-Free  Performance  Obtained 
with  4  of  the  Different  Types  of  SDFs . 
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5.  TRAINING  SET  SELECTION 

The  six  training  set  images  per  class  used  in  Table  2  were  50°  apart.  Those  in  Tfble  3 
were  30°  apart  and  from  one  side  of  the  ship  only.  This  ad  hoc  trainina  set  selection  ap¬ 
proach  is  quite  compatible  with  the  intuition  associated  with  such  a  pattern  recognition 
problem  (i.e.,  ship  images  appear  in  general  to  contain  little  new  information  if  viewed 
from  one  side  rather  than  the  other).  However,  in  other  cases,  more  automated  training  set 
selection  techniques  are  required.  Two  such  technicues  are  briefly  described  and  initial 
test  and  performance  data  obtained  using  them  are  presented  in  this  section. 

The  purpose  of  the  training  set  for  each  object  class  is  to  represent  each  object  type  by 
the  most  valid  statistical  representation  from  the  available  imagery.  The  SDF  algorithm  it¬ 
self  provides  the  inter-class  discrimination  reauired.  Thus,  our  concern  in  the  automatic 
selection  of  a  training  set  should  address  selecting  a  small  set  of  images  from  the  available 
image  data  base  such  that  they  represent  a  sufficiently  valid  statistical  representation  of 
each  object.  We  have  considered  two  ways  to  achieve  this:  a  correlation-subspace  and  an 
eigenvector  subspace  selection  technique. 

In  the  first  technique,  we  initially  pick  one  image  xij  (e.g.,  the  bow  view  of  a  ship). 

As  the  second  image,  we  select  that  image  from  the  remaining  ones  which  gives  the  minimum 
correlation  with  the  prior  image.  Such  an  image  contains  the  most  new  information  and  is 
the  best  one  to  select  for  our  training  set.  We  continue  this  technique  and  at  each  succes¬ 
sive  step,  we  pick  as  our  new  image  the  one  with  the  minimum  projection  on  the  sum  of  the 
previously  chosen  training  set  images.  With  proper  image  normalization  and  with  the  use  of 
a  modified  Gramm-Schmidt  technique,  we  can  thus  select  the  P  best  training  set  images  to  use. 
The  value  of  the  correlation  between  a  new  image  and  the  sum  of  the  prior  training  set  images  provi  des  a  measure  of  the 
new  data  added  by  a  giver  additional  training  set  image.  This  procedure  is  used  for  each  object  class  separately. 
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In  the  second  technique,  we  forin  the  correlation  matrix  P  for  each  object  class  and  cal¬ 
culate  its  eigenvalues  and  eigenvectors,  ’’e  select  the  dominant  eigenvectors  and  calculate 
the  projections  of  all  images  onto  these  dominant  eigenvectors.  Those  images  with  the  larg¬ 
est  projections  best  represent  the  object  class  and  are  thus  included  in  the  training  set. 

The  eigenvalue  is  used  to  determine  the  number  of  images  to  be  selected  for  each  dominant- 
eigenvector  .  The  value  of  the  projection  of  a  new  image  on  a  new  dominant  eigenvector  de¬ 
termines  the  additional  information  present  if  this  image  is  included  in  the  training  set. 

The  results  obtained  with  this  technique  are  summarized  in  Table  4  and  compared  to  those 
obtained  earlier  using  our  ad  hoc  training  set  selection  technique.  As  seen,  our  ad  hoc 
selection  of  6  images  gave  poorer  performance  than  did  both  of  the  new  methods  noted  in  this 
section.  Essentially  perfect  performance  (0  or  1  error  out  of  144  images)  was  obtained  using 
our  new  training  set  selection  methods.  As  the  last  entry  in  Tabic  '  shows,  a  successful 
single  SDF  for  the  4-class  problem  was  realized  using  such  training  set  methods.  The  per¬ 
formance  of  these  new  SDFs  in  the  presence  of  noise  was  found  to  be  comparable  to  those  ob¬ 
tained  with  the  previous  data.  An  average  of  only  4  or  5  errors  was  obtained  (versus  8-10 
errors  for  the  cases  noted  in  Tables  2  and  3) . 


TABLE  4.  Comparison  of  the  Performance  of  Synthetic  Discriminant 
Functions  Using  Ad  Hoc  and  Automated  Techniques  (Cor¬ 
relation  Subspace  and  Eigenvector  (e-v)  Subspace)  to 
Select  the  Image  Training  Sets. 
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6.  SUMMARY  AND  CONCLUSION  6.  SUMAPV  A yp  CONCLUSION 

In  this  paper,  we  have  described  a  new  technique  to  recognize  multiple  classes  of  objects 
in  the  face  of  geometrical  distortions.  Our  technique  uses  a  correlator  and  thus  allows 
multij le  object  recognition  and  excellent  performance  in  the  presence  of  structured  noise 
and  clutter.  The  latter  features  result  from  the  use  of  a  correlator  and  the  shift-invariance 
and  processing  gain  of  such  a  pattern  recognition  architecture.  The  ability  to  recognize 
distorted  object  views  and  to  discriminate  between  different  object  classes  is  achieved  by 
our  new  matched  spatial  filter  technique  using  a  matched  spatial  filter  of  a  synthetic  dis¬ 
criminant  function. 

Five  different  synthetic  discriminant  functions  have  been  df scribed  and  a  general  unified 
synthesis  procedure  to  form  each  off-line  usin'-  a  training  set  of  images  has  been  advanced. 
Initial  simulation  results  showed  excellent  "crformance  and  showed  the  robustness  of  this 
algorithm  in  the  presence  of  noise.  New  techrioues  to  select  the  training  set  for  such  a 
system  were  also  advanced  and  initial  per forr'rnce  results  were  .btained  and  were  compared  to 
those  obtained  for  other  ad  hoc  training  set  selection  techniaues.  Superior  performance  re¬ 
sulted  when  these  new  training  set-based  filters  were  employed. 
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1.  Night  Vision  and  Electro  Optics  Laboratory,  Ft.  Belvoir,  Virginia  - 
"Optical  Pattern  Recognition". 

2.  Night  Vision  and  Electro  Optics  Laboratory,  Ft.  Belvoir,  Virginia  - 
"Optical  Signal  Processing". 

3.  LIA  Conference,  Boston,  Massachusetts  -  "Realization  of  a  Sobel  Operator 
by  Coherent  Optical  Techniques". 

4.  LIA  Conference,  Boston,  Massachusetts  -  "A  Fisher  Discriminant  Approach  to 
Distortion-Invariant  Pattern  Recognition  Using  Autocorrelations”. 

5.  Ames  Building,  Washington,  D.C.  -  "Optical  Matrix-Vector  and  Optical 
Systolic  Array  Processors" 

October  1982: 

6.  Carnegie-Mellon  University,  Pittsburgh,  Pennsylvania,  Sophomore  Seminar  - 
"Optical  Data  Processing  at  Carnegie-Mellon  University". 

7.  TASC,  McLean,  Virginia  -  "General  Purpose  Optical  Processors". 

November  1982: 

8.  Hughes  Corporation,  Conoga  Park,  California  -  "Moment-Based  Hybrid  Optical/ 
Digital  Pattern  Recognition". 

9.  Hughes  Corporation,  Conoga  Park,  California  -  "SDFs  for  Distortion-Invariant 
Recognition  of  Multiple  Targets  in  Clutter". 

December  1982: 

10.  Eaton  Corporation,  Milwaukee,  Wisconsin  -  "Optical  Processing  Techniques  for 
Industrial  Inspection,  Automation  and  Pattern  Recognition". 

January  1983: 

11.  SPIE,  Los  Angeles,  California  -  "Optical  Linear  Algebra"  (Anjan  Ghosh). 
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April  1983: 

12.  IOCC  Conference,  Boston,  Massachusetts  -  "Guidelines  for  Efficient  Utili¬ 
zation  of  Optical  Systolic  Array  Processors". 

13.  IEEE,  Boston,  Massachusetts  -  "Optical  Data  Processing  for  Target  Recog¬ 
nition  and  Classification". 

14.  IOCC  Conference,  Boston,  Massachusetts  -  "Performance  of  Synthetic  Discrimi¬ 
nant  Functions  for  Infrared  Ship  Classification". 

May  1983: 

15.  Carnegie-Mellon  University,  Pittsburgh,  Pennsylvania,  Freshman  Seminar  - 
"Optical  Data  Processing". 

16.  Westinghouse  Corporation,  Baltimore,  Maryland  -  "Optical  Systolic  Processors 
for  Adaptive  Phased  Array  Radar". 

17.  Baltimore,  Maryland  -  "Optical  Data  Processing  Research  at  Carnegie-Mellon 
University". 

18.  CLEO  Conference,  Baltimore,  Maryland  -  "Advanced  Optical  Data  Processing 
Research" . 

19.  Rosslyn,  Virginia,  "Advanced  Object  Pattern  Recognition  in  Man-Made  Clutter" 

June  1983: 

20.  AGARD/NATO,  Bonn,  Germany  -  "Fundamentals  of  Optical  Pattern  Recognition". 

21.  AGARD/NATO,  Bonn,  Germany  -  "Advanced  Optical  Pattern  Recognition  Techniques 

22.  AGARD/NATO,  Bonn,  Germany  -  "Optical  Matrix-Vector  and  Systolic  Array 
Processing" . 

23.  Lebedev  Institute,  Moscow,  USSR  -  "Optical  Image,  Signal  and  Information 
Processing". 

24.  Latvian  Academy  of  Sciences,  Riga,  Latvia,  USSR  -  "Optical  Image  and  Data 
Processing" . 

25.  Institute  of  Physics,  Belyo,  Minsk,  USSR,  "Optical  Image  and  Information 
Processing". 

26.  Shuvalov  Institute,  Leningrad,  USSR  -  "Optical  Image,  Signal  and  Data 
Processing". 

27.  Ioffe  Institute,  Leningrad,  USSR  -  "Optical  Image  and  Data  Processing". 

28.  University  of  Kosova,  Pristina,  Yugoslavia  -  "Optical  Image,  Signal  and 
Data  Processing". 
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July  1983: 

29.  Instituto  de  Optiko,  Madrid,  Spain  -  "Optical  Image  and  Data  Processing". 
August  1983 : 

30.  SPIE,  San  Diego,  California  -  "Direct  and  Indirect  Optical  Solutions  to 
Linear  Algebraic  Equations:  Error  Source  Modeling". 

31.  SPIE,  San  Diego,  California  -  "Linear  Algebra  Techniques  for  Pattern  Rec¬ 
ognition:  Feature  Extraction  Case  Studies". 

32.  SPIE,  San  Diego,  California  -  "Shift-Invariant  and  Distortion-Invariant 
Object  Recognition". 


16.3  THESES  SUPPORTED  BY  AFOSR  FUNDING  (SEPTEMBER  1980  -  SEPTEMBER  1983) 

1.  Hiroyasu  Murakami,  M.S.  Dissertation,  "Matched  Filter  Statistical  Correlator" 
(February  1981). 

2.  Saulius  Eiva,  M.S.  Dissertation,  "Image  Quality  Effects  in  Optical  Corre¬ 
lators"  (May  1981). 

3.  Charles  Hester,  PhD  Dissertation,  "Synthetic  Filters  for  Multi-Class  Pattern 
Recognition"  (May  1981). 

4.  Yair  Barniv,  PhD  Dissertation,  "Multi-Sensor  Image  Registration"  (May  1981). 

5.  Mark  Carlotto,  PhD  Dissertation,  "Iterative  Electro-Optic  Matrix  Processor" 
(May  1981). 

6.  Andrew  Sexton,  M.S.  Dissertation,  "Digital  Analysis  of  Space-Variant  Optical 
Processors"  (July  1981). 

7.  Bernard  Szymanski,  M.S.  Dissertation,  "A  Computer-Controlled  Film  Recorder 
for  Optical  Processing”  (July  1983). 

8.  John  Lycas,  M.S.  Dissertation,  "An  Optical  Implementation  of  the  Extended 
Kalman  Filter"  (September  1983) . 

9.  Vinod  Sharma,  "Synthetic  Discriminant  Functions"  (PhD  Expected  in  December 
1983). 

10.  Warren  Allmond,  "New  Holographic  Optical  Elements  and  Architectures"  (M.S.  ' 

Expected  in  1984). 

11.  Eugene  Pochapsky,  "Digital  Preprocessing  and  Simulation  for  Optical  Pattern 
Recognition"  (M.S.  Expected  in  1984). 

12.  R.  Lee  Cheatham,  "Optical  Moment-Based  Pattern  Recognition"  (PhD  Expected 
in  1984). 


13.  Anjan  Ghosh,  "Linear  Algebra  System  Performance  and  Analysis"  (PhD  Expected 
in  1984). 

14.  G.  Kashipati,  "Synthetic  Discriminant  Functions"  (M.S.  Expected  in  1984). 

15.  Bruce  Thomas,  "Moments  for  Distortion  Measurement"  (M.S.  Expected  in  1984). 

16.  William  Rozzi,  "Synthetic  Discriminant  Functions"  (M.S.  Expected  in  1984). 

17.  Wen-Thong  Chang,  "Optical  Pattern  Recognition"  (PhD  Expected  in  1985). 

16.4  PATENT  DISCLOSURES  (SEPTEMBER  1980  -  SEPTEMBER  1983) 

1.  Multiple-Invariant  Space-Variant  Pattern  Recognition  System. 

2.  Pattern  Recognition  by  Invariant  Moments. 

3.  Synthetic  Discriminant  Functions  for  Multi-Class  Pattern  Recognition. 

4.  Equalization  and  Coherence  Measure  Correlator. 

5.  Multi-Variant  Technique  for  Multi-Class  Pattern  Recognition. 
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